Jul 29 01:10:21.618706 ip-10-0-141-49 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 01:10:21.618717 ip-10-0-141-49 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 01:10:21.618726 ip-10-0-141-49 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 01:10:21.619192 ip-10-0-141-49 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 01:10:31.833968 ip-10-0-141-49 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 01:10:31.833985 ip-10-0-141-49 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1afb9e6142a34a168817e52930f9036f --
Jul 29 01:12:24.121988 ip-10-0-141-49 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 01:12:24.545641 ip-10-0-141-49 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 01:12:24.545641 ip-10-0-141-49 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 01:12:24.545641 ip-10-0-141-49 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 01:12:24.545641 ip-10-0-141-49 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 01:12:24.545641 ip-10-0-141-49 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 01:12:24.547935 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.547846    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 01:12:24.552034 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552014    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.552034 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552031    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.552034 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552037    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552042    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552046    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552051    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552055    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552060    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552064    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552068    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552073    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552079    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552085    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552089    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552094    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552098    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552102    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552107    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552112    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552115    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552120    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552125    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.552216 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552129    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552133    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552137    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552141    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552145    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552150    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552154    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552159    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552163    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552167    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552171    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552175    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552179    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552186    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552191    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552195    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552200    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552204    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552209    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.553066 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552214    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552218    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552222    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552226    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552230    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552234    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552238    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552243    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552249    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552253    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552258    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552262    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552266    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552270    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552275    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552279    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552283    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552288    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552293    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552297    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.553872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552301    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552306    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552310    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552314    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552318    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552323    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552328    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552334    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552338    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552343    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552347    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552351    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552357    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552362    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552366    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552370    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552375    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552379    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552383    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.554668 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552387    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552394    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552400    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552404    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552409    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552414    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552419    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552424    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552428    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552433    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552438    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552443    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552447    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552451    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552455    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552462    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552467    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552472    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.552477    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.555456 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553661    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553672    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553677    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553681    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553686    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553690    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553695    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553699    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553705    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553726    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553731    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553735    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553739    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553744    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553748    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553753    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553757    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553761    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553766    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553770    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.556002 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553775    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553780    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553784    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553788    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553792    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553796    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553801    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553805    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553810    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553814    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553818    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553822    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553826    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553831    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553836    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553841    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553845    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553857    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553861    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553865    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.556488 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553870    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553875    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553879    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553883    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553887    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553890    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553896    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553903    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553907    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553911    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553916    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553920    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553924    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553929    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553933    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553937    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553941    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553945    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553950    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553955    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.556998 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553959    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553963    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553967    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553971    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553975    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553981    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553986    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553990    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553995    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.553999    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554003    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554008    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554012    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554016    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554021    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554028    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554036    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554043    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554047    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.557478 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554052    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554056    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554061    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554065    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554069    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554073    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554077    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554081    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554085    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554089    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554094    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554098    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554102    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554106    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554110    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554114    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554119    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554123    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554128    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.557985 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.554132    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555533    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555545    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555549    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555551    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555555    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555558    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555561    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555563    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555566    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555569    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555572    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555574    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555577    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555580    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555583    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555585    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555588    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555590    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.558437 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555593    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555598    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555602    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555605    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555608    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555611    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555614    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555617    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555619    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555622    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555625    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555627    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555630    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555633    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555639    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555642    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555646    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555648    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555651    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.558920 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555653    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555656    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555659    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555661    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555664    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555666    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555669    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555673    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555677    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555681    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555685    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555688    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555691    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555694    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555697    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555700    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555703    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555705    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555708    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.559415 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555725    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555728    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555731    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555733    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555736    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555739    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555742    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555744    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555747    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555750    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555753    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555755    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555759    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555762    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555764    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555767    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555770    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555773    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555775    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.559898 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555777    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555780    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555783    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555785    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555788    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555790    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555793    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555796    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555799    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555801    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555804    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555806    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555809    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555811    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555814    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555816    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555819    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555821    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555824    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555827    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.560348 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555830    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555832    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555835    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.555838    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555905    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555913    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555919    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555923    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555928    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555933    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555940    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555946    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555949    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555953    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555956    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555959    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555962    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555965    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555968    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555972    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555976    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555979    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555983    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555986    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 01:12:24.560903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555989    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555992    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.555995    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556000    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556003    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556006    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556009    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556012    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556016    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556019    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556022    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556025    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556035    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556038    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556041    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556044    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556047    2564 flags.go:64] FLAG: --enable-server="true"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556050    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556054    2564 flags.go:64] FLAG: --event-burst="100"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556072    2564 flags.go:64] FLAG: --event-qps="50"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556076    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556079    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556083    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556086    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 01:12:24.561482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556089    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556092    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556095    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556098    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556102    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556105    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556108    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556111    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556114    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556116    2564 flags.go:64] FLAG: --feature-gates=""
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556120    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556123    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556126    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556129    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556132    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556135    2564 flags.go:64] FLAG: --help="false"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556138    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556141    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556146    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556149    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556152    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556155    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556158    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 01:12:24.562088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556161    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556164    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556167    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556169    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556172    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556175    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556178    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556181    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556184    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556187    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556189    2564 flags.go:64] FLAG: --lock-file=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556192    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556195    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556198    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556204    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556207    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556210    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556212    2564 flags.go:64] FLAG: --logging-format="text"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556215    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556218    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556221    2564 flags.go:64] FLAG: --manifest-url=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556224    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556228    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556231    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556235    2564 flags.go:64] FLAG: --max-pods="110"
Jul 29 01:12:24.562686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556238    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556240    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556245    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556248    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556251    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556253    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556256    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556266    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556269    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556272    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556275    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556278    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556283    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556286    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556289    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556291    2564 flags.go:64] FLAG: --port="10250"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556294    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556297    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bcc933a55f8caba1"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556300    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556305    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556308    2564 flags.go:64] FLAG: --register-node="true"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556310    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556314    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556317    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 29 01:12:24.563322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556320    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556323    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556327    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556330    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556333    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556336    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556338    2564 flags.go:64] FLAG: --runonce="false"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556341    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556344    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556347    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556350    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556354    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556357    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556360    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556363    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556365    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556368    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556371    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556374    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556377    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556380    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556382    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556388    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556392    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556394    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 01:12:24.563920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556398    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556401    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556406    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556408    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556411    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556414    2564 flags.go:64] FLAG: --v="2"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556418    2564 flags.go:64] FLAG: --version="false"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556428    2564 flags.go:64] FLAG: --vmodule=""
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556432    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.556435    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556573    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556578    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556582    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556585    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556588    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556591    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556594    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556597    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556601    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556604    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556606    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.564560 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556609    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556611    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556614    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556616    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556619    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556622    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556624    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556627    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556629    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556632    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556635    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556637    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556640    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556645    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556647    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556650    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556652    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556655    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556657    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556665    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.565091 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556668    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556670    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556673    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556675    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556678    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556680    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556683    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556685    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556688    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556692    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556694    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556697    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556699    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556702    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556705    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556707    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556722    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556725    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556728    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556731    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556733    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.565571 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556736    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556738    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556741    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556744    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556748    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556750    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556753    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556755    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556759    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556763    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556766    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556774    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556777    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556780    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556782    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556785    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556787    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556790    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556793    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.566114 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556795    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556799    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556802    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556804    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556807    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556810    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556812    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556815    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556817    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556821    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556825    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556828    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556830    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556833    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556835    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556838    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556841    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556845    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556847    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.566617 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556850    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556852    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556855    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556857    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556860    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556862    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556866    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556869    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.556871    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.557453    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564024    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564040    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564059    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564065    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 01:12:24.567121 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564121    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564127    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564131    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564134    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564138    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564140    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564143    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564146    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564148    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564151    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564153    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564156    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564159    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564161    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564164    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564167    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564169    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564172    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564174    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564177    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.567475 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564180    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564183    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564185    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564188    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564190    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564193    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564195    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564198    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564200    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564204    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564207    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564209    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564212    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564214    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564217    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564220    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564222    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564225    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564228    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564230    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.567979 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564233    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564235    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564238    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564240    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564243    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564245    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564248    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564251    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564253    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564256    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564258    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564261    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564263    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564266    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564269    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564271    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564274    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564278    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564282    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.568464 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564285    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564287    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564291    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564295    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564299    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564302    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564305    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564307    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564310    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564312    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564315    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564318    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564320    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564322    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564325    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564328    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564330    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564333    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564335    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564338    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.568945 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564341    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564343    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564346    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564348    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564351    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564353    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564356    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564359    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564361    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564364    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564366    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564369    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564371    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564374    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564377    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564380    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564383    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564386    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564388    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.569433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564391    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564395    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564506    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564510    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564513    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564516    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564518    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564521    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564525    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564527    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564530    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564532    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564535    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564537    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564540    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 01:12:24.569910 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564543    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564545    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564548    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564550    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564553    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564555    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564558    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564561    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564564    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564566    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564569    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564572    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564575    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564578    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564581    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564584    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564586    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564589    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564592    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564594    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 01:12:24.570285 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564596    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564599    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564601    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564604    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564607    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564609    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564612    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564616    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564618    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564621    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564623    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564626    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564628    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564631    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564633    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564636    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564638    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564641    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564645    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 01:12:24.570879 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564648    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564651    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564654    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564656    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564660    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564663    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564665    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564668    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564671    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564674    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564677    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564681    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564684    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564687    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564690    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564693    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564696    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564699    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564701    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564704    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 01:12:24.571330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564707    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564728    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564732    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564735    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564738    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564740    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564743    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564746    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564749    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564751    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564754    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564756    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564759    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564761    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564764    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564766    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564769    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564771    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564774    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564776    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 01:12:24.571852 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564779    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564782    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564785    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564788    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564790    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564793    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:24.564795    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564800    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.564907    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.566838    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.568098    2564 server.go:1019] "Starting client certificate rotation"
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.568198    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.568227    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 01:12:24.572330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.570516    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 01:12:24.574917 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.574902    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 01:12:24.586915 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.586896    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 29 01:12:24.589312 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.589293    2564 log.go:25] "Validated CRI v1 image API"
Jul 29 01:12:24.590197 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.590173    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 01:12:24.594886 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.594868    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 01:12:24.595209 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.595189    2564 fs.go:135] Filesystem UUIDs: map[625027c1-939b-4060-b1e5-e5fe48e2733b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8c337efe-018d-494f-8fd8-69b5503c88f9:/dev/nvme0n1p3]
Jul 29 01:12:24.595273 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.595209    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 01:12:24.600295 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.600192    2564 manager.go:216] Machine: {Timestamp:2026-07-29 01:12:24.598795098 +0000 UTC m=+0.355111020 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100443 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2c8e82d680024ba2e4c815909c77 SystemUUID:ec2d2c8e-82d6-8002-4ba2-e4c815909c77 BootID:1afb9e61-42a3-4a16-8817-e52930f9036f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b7:1f:b7:83:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b7:1f:b7:83:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:9a:aa:68:ff:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 01:12:24.600295 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.600293    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 01:12:24.600409 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.600398    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 01:12:24.603560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.603538    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 01:12:24.603703 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.603563    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-49.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 01:12:24.604327 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.604317    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 01:12:24.604358 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.604329    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 01:12:24.604358 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.604336    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 01:12:24.605041 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.605031    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 01:12:24.605841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.605832    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 01:12:24.607516 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607505    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 01:12:24.607634 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607624    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 01:12:24.607676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607652    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 01:12:24.607676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607662    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 01:12:24.607676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607672    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 01:12:24.607801 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607681    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 29 01:12:24.607801 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.607694    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 01:12:24.608092 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.608079    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 01:12:24.608131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.608101    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 01:12:24.610245 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.610221    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 01:12:24.612193 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.612178    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 01:12:24.612259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.612208    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 01:12:24.613700 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613689    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613704    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613722    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613730    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613740    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613745    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613753    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613758    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613764    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 01:12:24.613767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613769    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 01:12:24.614053 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613777    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 01:12:24.614053 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.613783    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 01:12:24.614629 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.614614    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 01:12:24.614629 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.614629    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 01:12:24.615776 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.615756    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 01:12:24.615776 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.615767    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 01:12:24.615893 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.615877    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-49.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 01:12:24.617809 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.617797    2564 server.go:1268] "Started kubelet"
Jul 29 01:12:24.617918 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.617887    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 01:12:24.617979 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.617941    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 01:12:24.618045 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.618022    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 01:12:24.618745 ip-10-0-141-49 systemd[1]: Started Kubernetes Kubelet.
Jul 29 01:12:24.619516 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.619496    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 01:12:24.619595 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.619572    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 01:12:24.623324 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.622227    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-49.ec2.internal.18c69d901edf2570  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-49.ec2.internal,UID:ip-10-0-141-49.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-49.ec2.internal,},FirstTimestamp:2026-07-29 01:12:24.6177724 +0000 UTC m=+0.374088320,LastTimestamp:2026-07-29 01:12:24.6177724 +0000 UTC m=+0.374088320,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-49.ec2.internal,}"
Jul 29 01:12:24.623446 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.623430    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 01:12:24.626412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.626389    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 01:12:24.626491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.626398    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 01:12:24.626987 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.626970    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 01:12:24.627092 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.626978    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 01:12:24.627176 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627166    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 01:12:24.627373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627345    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 01:12:24.627373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627363    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 01:12:24.627487 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.627436    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:24.627487 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627465    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 01:12:24.627583 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627492    2564 factory.go:55] Registering systemd factory
Jul 29 01:12:24.627583 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627501    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 29 01:12:24.627768 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627751    2564 factory.go:153] Registering CRI-O factory
Jul 29 01:12:24.627768 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627769    2564 factory.go:223] Registration of the crio container factory successfully
Jul 29 01:12:24.627894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627790    2564 factory.go:103] Registering Raw factory
Jul 29 01:12:24.627894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.627802    2564 manager.go:1196] Started watching for new ooms in manager
Jul 29 01:12:24.628484 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.628460    2564 manager.go:349] Starting recovery of all containers
Jul 29 01:12:24.637662 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.637629    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfd2h"
Jul 29 01:12:24.637773 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.637693    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 01:12:24.637831 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.637791    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 01:12:24.641224 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.641208    2564 manager.go:354] Recovery completed
Jul 29 01:12:24.644599 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.644583    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfd2h"
Jul 29 01:12:24.645235 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.645224    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.647075 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647060    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.647137 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647087    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.647137 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647096    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.647665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647649    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 01:12:24.647665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647662    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 01:12:24.647811 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647675    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.647811 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.647677    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 01:12:24.648384 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.648367    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.648444 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.648411    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.648444 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.648423    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.649246 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.649233    2564 policy_none.go:49] "None policy: Start"
Jul 29 01:12:24.649246 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.649248    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 01:12:24.649333 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.649258    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 01:12:24.650453 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.650442    2564 policy_none.go:47] "Start"
Jul 29 01:12:24.694371 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694238    2564 manager.go:337] "Starting Device Plugin manager"
Jul 29 01:12:24.694472 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.694389    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 01:12:24.694472 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694401    2564 server.go:87] "Starting device plugin registration server"
Jul 29 01:12:24.694638 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694625    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 01:12:24.694695 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694640    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 01:12:24.694760 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694743    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 01:12:24.694836 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694821    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 01:12:24.694836 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.694836    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 01:12:24.695323 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.695304    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 01:12:24.695389 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.695358    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:24.784313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.784276    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 01:12:24.785500 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.785483    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 01:12:24.785586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.785522    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 01:12:24.785586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.785549    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 01:12:24.785586 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.785581    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 01:12:24.787268 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.787247    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:24.795176 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.795160    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.795992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.795933    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.795992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.795960    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.795992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.795969    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.795992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.795990    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.808097 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.808083    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.808154 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.808104    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-49.ec2.internal\": node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:24.827177 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.827158    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:24.886103 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.886068    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"]
Jul 29 01:12:24.886521 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.886508    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.887967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.887950    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.888037 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.887978    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.888037 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.887988    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.889159 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889147    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.889276 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.889326 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889300    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.889811 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889791    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.889887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889817    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.889887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889826    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.889887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889791    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.889986 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889897    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.889986 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.889908    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.890981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.890964    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.891067 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.890987    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 01:12:24.891643 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.891627    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 01:12:24.891740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.891648    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 01:12:24.891740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:24.891658    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeHasSufficientPID"
Jul 29 01:12:24.906255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.906238    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-49.ec2.internal\" not found" node="ip-10-0-141-49.ec2.internal"
Jul 29 01:12:24.911573 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:24.911555    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-49.ec2.internal\" not found" node="ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.252548 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.252533    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.252680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.252664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.252723 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.252687    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3616ae90a98efffd0c217d8bdda2d07e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-49.ec2.internal\" (UID: \"3616ae90a98efffd0c217d8bdda2d07e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.252723 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.252705    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.352634 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.352604    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.353734 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.353802 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353742    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3616ae90a98efffd0c217d8bdda2d07e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-49.ec2.internal\" (UID: \"3616ae90a98efffd0c217d8bdda2d07e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.353802 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.353891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.353891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353829    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3616ae90a98efffd0c217d8bdda2d07e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-49.ec2.internal\" (UID: \"3616ae90a98efffd0c217d8bdda2d07e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.353891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.353836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2184daae0660a411ecef5fddf1a47bfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal\" (UID: \"2184daae0660a411ecef5fddf1a47bfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.453342 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.453288    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.533590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.533560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.539511 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.539496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:26.553751 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.553732    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.654184 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.654162    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.754646 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.754597    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.758311 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.758296    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:26.855321 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.855297    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.892523 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.892505    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 01:12:26.892891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.892620    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 01:12:26.892891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.892633    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 01:12:26.955434 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:26.955409    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:26.974503 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.974464    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 01:07:25 +0000 UTC" deadline="2028-01-26 23:18:21.222645376 +0000 UTC"
Jul 29 01:12:26.974503 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:26.974489    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13126h5m54.248158824s"
Jul 29 01:12:27.049486 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:27.049449    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2184daae0660a411ecef5fddf1a47bfc.slice/crio-3162e5c2fdee67cf6cc0868aeb7166aa3c42d3c8a8e67bede51882b79ab0da52 WatchSource:0}: Error finding container 3162e5c2fdee67cf6cc0868aeb7166aa3c42d3c8a8e67bede51882b79ab0da52: Status 404 returned error can't find the container with id 3162e5c2fdee67cf6cc0868aeb7166aa3c42d3c8a8e67bede51882b79ab0da52
Jul 29 01:12:27.056280 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:27.056265    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:27.057287 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.057273    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:12:27.065848 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:27.065829    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3616ae90a98efffd0c217d8bdda2d07e.slice/crio-43db82a6cea15299b44dd8e349cd50e4fe749f3a7ca2f692cdb21ff70bfdf89e WatchSource:0}: Error finding container 43db82a6cea15299b44dd8e349cd50e4fe749f3a7ca2f692cdb21ff70bfdf89e: Status 404 returned error can't find the container with id 43db82a6cea15299b44dd8e349cd50e4fe749f3a7ca2f692cdb21ff70bfdf89e
Jul 29 01:12:27.112812 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.112767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal" event={"ID":"3616ae90a98efffd0c217d8bdda2d07e","Type":"ContainerStarted","Data":"43db82a6cea15299b44dd8e349cd50e4fe749f3a7ca2f692cdb21ff70bfdf89e"}
Jul 29 01:12:27.113747 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.113729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal" event={"ID":"2184daae0660a411ecef5fddf1a47bfc","Type":"ContainerStarted","Data":"3162e5c2fdee67cf6cc0868aeb7166aa3c42d3c8a8e67bede51882b79ab0da52"}
Jul 29 01:12:27.157024 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:27.156994    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-49.ec2.internal\" not found"
Jul 29 01:12:27.216993 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.216972    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:27.217066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.217008    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:27.252006 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.251972    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:27.264345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.264330    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 01:12:27.265449 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.265437    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal"
Jul 29 01:12:27.273711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.273699    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 01:12:27.933010 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.932983    2564 apiserver.go:52] "Watching apiserver"
Jul 29 01:12:27.942497 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.942470    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 01:12:27.942998 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.942974    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-nkp4z","kube-system/global-pull-secret-syncer-sh9cm","kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qsvht","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal","openshift-multus/multus-v6lrj","openshift-multus/network-metrics-daemon-2tb8c","openshift-network-diagnostics/network-check-target-qw64x","kube-system/konnectivity-agent-6kth6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979","openshift-dns/node-resolver-tqr8t","openshift-image-registry/node-ca-2l955","openshift-multus/multus-additional-cni-plugins-qr45k","openshift-network-operator/iptables-alerter-h8qrf"]
Jul 29 01:12:27.944692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.944666    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:27.944803 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:27.944749    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:27.945798 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.945779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:27.947346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.947328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:27.948334 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.948317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9wslv\""
Jul 29 01:12:27.948476 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.948453    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.948573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.948558    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.948696 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.948682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.950225 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.950207    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:27.950320 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:27.950271    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:27.950862 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.950593    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 01:12:27.950862 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.950657    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jqfq4\""
Jul 29 01:12:27.950862 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.950740    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.950862 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.950738    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.952174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.951386    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 01:12:27.952174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.951406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.952174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.951421    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.952174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.951435    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fp5nd\""
Jul 29 01:12:27.958384 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.958367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.959559 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.959539    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:27.960874 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.960845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4m5k\" (UniqueName: \"kubernetes.io/projected/9447e1e0-a540-4af2-b582-6628c24a78b6-kube-api-access-t4m5k\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.960954 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.960895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-cni-binary-copy\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.960954 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.960926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-k8s-cni-cncf-io\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961074 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.960955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxphf\" (UniqueName: \"kubernetes.io/projected/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-kube-api-access-pxphf\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:27.961074 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.960992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd96a982-23af-4213-8fd8-27f14ef2cb2c-host-slash\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:27.961074 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-system-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961074 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961045    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.961284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-netns\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-kubelet\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-daemon-config\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961227    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-multus-certs\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961469 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:27.961469 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961435    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 01:12:27.961469 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.961612 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.961612 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:27.961612 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-bin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961739 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-conf-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.961792 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-modprobe-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.961837 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-run\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.961837 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961829    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-sys\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.961926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-lib-modules\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.961976 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-tuned\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.962028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2v7k4\""
Jul 29 01:12:27.962028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961998    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.961979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-socket-dir-parent\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-hostroot\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962549    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962629    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-etc-kubernetes\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlbgm\" (UniqueName: \"kubernetes.io/projected/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-kube-api-access-hlbgm\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m55b\" (UniqueName: \"kubernetes.io/projected/4686ca3b-0f27-45c9-92bc-c36426580d85-kube-api-access-5m55b\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd96a982-23af-4213-8fd8-27f14ef2cb2c-iptables-alerter-script\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-conf\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962923    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-os-release\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-multus\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.962997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-tmp\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.963025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.963295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-tmp-dir\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.963438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-kubernetes\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.963464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-host\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.963640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.963492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-cnibin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:27.965230 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965210    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:27.965318 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-hosts-file\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:27.965318 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bnhq\" (UniqueName: \"kubernetes.io/projected/fd96a982-23af-4213-8fd8-27f14ef2cb2c-kube-api-access-4bnhq\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:27.965421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysconfig\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.965421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-systemd\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.965421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-var-lib-kubelet\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:27.965562 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.965472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:27.966506 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:27.965293    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:27.966506 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.966245    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bv5fs\""
Jul 29 01:12:27.967956 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.966662    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:27.968253 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.968210    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:27.968339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.968262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 01:12:27.968461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.968444    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 01:12:27.968946 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.968698    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-49mch\""
Jul 29 01:12:27.969928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.969671    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6v4zf\""
Jul 29 01:12:27.969928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.969710    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.969928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.969678    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 01:12:27.969928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.969894    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.971166 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.971015    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 01:12:27.971926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.971188    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 01:12:27.971926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.971260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.971926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.971698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:27.972717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.972344    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bs558\""
Jul 29 01:12:27.972717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.972463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 01:12:27.972717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.972571    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 01:12:27.972717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.972645    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.974208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.974038    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 01:12:27.974208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.974079    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 01:12:27.974368 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.974300    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 01:12:27.974368 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.974323    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fk7s9\""
Jul 29 01:12:27.986481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.986417    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7shdd"
Jul 29 01:12:27.993908 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:27.993889    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7shdd"
Jul 29 01:12:28.053330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.053308    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 01:12:28.066205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-kubelet-config\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnhq\" (UniqueName: \"kubernetes.io/projected/fd96a982-23af-4213-8fd8-27f14ef2cb2c-kube-api-access-4bnhq\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-systemd\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-cni-binary-copy\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-daemon-config\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-multus-certs\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.066332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-systemd\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-registration-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-systemd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxphf\" (UniqueName: \"kubernetes.io/projected/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-kube-api-access-pxphf\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-netns\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-netns\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovn-node-metrics-cert\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066506    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-var-lib-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.066589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-ovn\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-sys\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-tuned\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-socket-dir-parent\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-multus-certs\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066928    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-daemon-config\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-hostroot\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-sys\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.066997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-socket-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.067047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31ed3c06-7825-4d83-9da3-43877fb9780f-host\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg7qb\" (UniqueName: \"kubernetes.io/projected/31ed3c06-7825-4d83-9da3-43877fb9780f-kube-api-access-vg7qb\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-hostroot\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlbgm\" (UniqueName: \"kubernetes.io/projected/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-kube-api-access-hlbgm\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067122    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-netns\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-device-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067249    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-node-log\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-os-release\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.067623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067623    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cnibin\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-log-socket\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99zkp\" (UniqueName: \"kubernetes.io/projected/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-kube-api-access-99zkp\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-conf-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-dbus\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-tmp-dir\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067789    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-kubernetes\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-cnibin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-k8s-cni-cncf-io\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-etc-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-hosts-file\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysconfig\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-var-lib-kubelet\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4m5k\" (UniqueName: \"kubernetes.io/projected/9447e1e0-a540-4af2-b582-6628c24a78b6-kube-api-access-t4m5k\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.067997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t49vq\" (UniqueName: \"kubernetes.io/projected/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-kube-api-access-t49vq\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068024    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkm98\" (UniqueName: \"kubernetes.io/projected/28e471cd-bab7-487b-9195-cb4e51d69100-kube-api-access-tkm98\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-socket-dir-parent\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.068177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-bin\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd96a982-23af-4213-8fd8-27f14ef2cb2c-host-slash\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-system-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-kubelet\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-netd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-modprobe-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-run\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-lib-modules\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-etc-kubernetes\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m55b\" (UniqueName: \"kubernetes.io/projected/4686ca3b-0f27-45c9-92bc-c36426580d85-kube-api-access-5m55b\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986c3fc6-5c33-494b-8e89-0483e1ddefc7-agent-certs\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31ed3c06-7825-4d83-9da3-43877fb9780f-serviceca\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4686ca3b-0f27-45c9-92bc-c36426580d85-cni-binary-copy\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068619    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-conf-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd96a982-23af-4213-8fd8-27f14ef2cb2c-host-slash\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-system-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-kubelet\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-modprobe-d\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-run\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.068808    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.068861    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:28.568843487 +0000 UTC m=+3.000725968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-lib-modules\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.068917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-etc-kubernetes\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-run-k8s-cni-cncf-io\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-os-release\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-os-release\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.069873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-hosts-file\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-kubelet\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-systemd-units\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-var-lib-kubelet\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-config\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd96a982-23af-4213-8fd8-27f14ef2cb2c-iptables-alerter-script\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-conf\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-multus\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-sys-fs\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986c3fc6-5c33-494b-8e89-0483e1ddefc7-konnectivity-ca\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-bin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-cnibin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-tmp\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-tmp-dir\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-kubernetes\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.070639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-multus\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysconfig\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.069762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-sysctl-conf\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.070958    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd96a982-23af-4213-8fd8-27f14ef2cb2c-iptables-alerter-script\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-etc-tuned\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-host\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-host-var-lib-cni-bin\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4686ca3b-0f27-45c9-92bc-c36426580d85-multus-cni-dir\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9447e1e0-a540-4af2-b582-6628c24a78b6-host\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-slash\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.071964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-env-overrides\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.072116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-script-lib\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.072546    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.072568    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.072581    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.072703    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:28.572685068 +0000 UTC m=+3.004567530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.074009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9447e1e0-a540-4af2-b582-6628c24a78b6-tmp\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.076752 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.076668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxphf\" (UniqueName: \"kubernetes.io/projected/1ea819b9-2b2d-4f81-aee1-d6e3636ee41a-kube-api-access-pxphf\") pod \"node-resolver-tqr8t\" (UID: \"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a\") " pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.078541 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.078484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bnhq\" (UniqueName: \"kubernetes.io/projected/fd96a982-23af-4213-8fd8-27f14ef2cb2c-kube-api-access-4bnhq\") pod \"iptables-alerter-h8qrf\" (UID: \"fd96a982-23af-4213-8fd8-27f14ef2cb2c\") " pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.080020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.079941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m55b\" (UniqueName: \"kubernetes.io/projected/4686ca3b-0f27-45c9-92bc-c36426580d85-kube-api-access-5m55b\") pod \"multus-v6lrj\" (UID: \"4686ca3b-0f27-45c9-92bc-c36426580d85\") " pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.081221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.081179    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlbgm\" (UniqueName: \"kubernetes.io/projected/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-kube-api-access-hlbgm\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:28.081698 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.081652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4m5k\" (UniqueName: \"kubernetes.io/projected/9447e1e0-a540-4af2-b582-6628c24a78b6-kube-api-access-t4m5k\") pod \"tuned-qsvht\" (UID: \"9447e1e0-a540-4af2-b582-6628c24a78b6\") " pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.172528 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-node-log\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cnibin\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-log-socket\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99zkp\" (UniqueName: \"kubernetes.io/projected/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-kube-api-access-99zkp\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-node-log\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-dbus\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.172664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-etc-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172684    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-dbus\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t49vq\" (UniqueName: \"kubernetes.io/projected/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-kube-api-access-t49vq\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkm98\" (UniqueName: \"kubernetes.io/projected/28e471cd-bab7-487b-9195-cb4e51d69100-kube-api-access-tkm98\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-bin\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-netd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986c3fc6-5c33-494b-8e89-0483e1ddefc7-agent-certs\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31ed3c06-7825-4d83-9da3-43877fb9780f-serviceca\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-etc-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.172959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.172990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-os-release\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-kubelet\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-systemd-units\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-config\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-sys-fs\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986c3fc6-5c33-494b-8e89-0483e1ddefc7-konnectivity-ca\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-slash\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-env-overrides\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-script-lib\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-kubelet-config\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-registration-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-systemd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-netns\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.173654 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173437    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovn-node-metrics-cert\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-var-lib-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-ovn\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-socket-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cnibin\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31ed3c06-7825-4d83-9da3-43877fb9780f-host\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg7qb\" (UniqueName: \"kubernetes.io/projected/31ed3c06-7825-4d83-9da3-43877fb9780f-kube-api-access-vg7qb\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-log-socket\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-device-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173911    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-bin\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.174461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.173961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-cni-netd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174100    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-system-cni-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-os-release\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174515    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-kubelet\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.174752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-systemd-units\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-config\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-var-lib-openvswitch\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175373    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/31ed3c06-7825-4d83-9da3-43877fb9780f-host\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175501    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-socket-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-ovn\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-device-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.175602    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:28.176427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-slash\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.175652    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:28.675635213 +0000 UTC m=+3.107517675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-sys-fs\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28e471cd-bab7-487b-9195-cb4e51d69100-registration-dir\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9438ab0c-aec4-41b1-b091-e541c119b72c-kubelet-config\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-run-systemd\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.175977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-host-run-netns\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.176012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986c3fc6-5c33-494b-8e89-0483e1ddefc7-konnectivity-ca\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.176046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-env-overrides\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.176070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovnkube-script-lib\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.177229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.176319    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/31ed3c06-7825-4d83-9da3-43877fb9780f-serviceca\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.178610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.178589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986c3fc6-5c33-494b-8e89-0483e1ddefc7-agent-certs\") pod \"konnectivity-agent-6kth6\" (UID: \"986c3fc6-5c33-494b-8e89-0483e1ddefc7\") " pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.180830 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.180808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-ovn-node-metrics-cert\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.182518 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.182471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t49vq\" (UniqueName: \"kubernetes.io/projected/6741d76d-6d73-4dad-b5ab-2cb9f3525cfd-kube-api-access-t49vq\") pod \"multus-additional-cni-plugins-qr45k\" (UID: \"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd\") " pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.183281 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.183230    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkm98\" (UniqueName: \"kubernetes.io/projected/28e471cd-bab7-487b-9195-cb4e51d69100-kube-api-access-tkm98\") pod \"aws-ebs-csi-driver-node-dn979\" (UID: \"28e471cd-bab7-487b-9195-cb4e51d69100\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.184833 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.184814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99zkp\" (UniqueName: \"kubernetes.io/projected/530ed9b3-f52e-448f-a9a5-3c92dfb00ff6-kube-api-access-99zkp\") pod \"ovnkube-node-nkp4z\" (UID: \"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.185134 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.185113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg7qb\" (UniqueName: \"kubernetes.io/projected/31ed3c06-7825-4d83-9da3-43877fb9780f-kube-api-access-vg7qb\") pod \"node-ca-2l955\" (UID: \"31ed3c06-7825-4d83-9da3-43877fb9780f\") " pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.214310 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.214291    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:28.271790 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.271763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tqr8t"
Jul 29 01:12:28.277742 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.277723    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8qrf"
Jul 29 01:12:28.286038 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.286020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qsvht"
Jul 29 01:12:28.292988 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.292969    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6lrj"
Jul 29 01:12:28.299977 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.299957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:28.307531 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.307509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qr45k"
Jul 29 01:12:28.315232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.315215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979"
Jul 29 01:12:28.321959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.321935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:28.328776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.328757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2l955"
Jul 29 01:12:28.577578 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.577496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:28.577578 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.577556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:28.577774 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577661    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:28.577774 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577708    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:28.577774 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577729    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:28.577774 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577737    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:28.577774 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577714    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:29.577701445 +0000 UTC m=+4.009583902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:28.577961 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.577787    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:29.577774322 +0000 UTC m=+4.009656780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:28.678334 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.678302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:28.678504 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.678439    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:28.678504 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:28.678496    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:29.678481478 +0000 UTC m=+4.110363945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:28.994383 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:28.994350    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4686ca3b_0f27_45c9_92bc_c36426580d85.slice/crio-8ea96066e5ec82e998b8e9c47eb2a3afdefa5cadfcc09b930dccf9f834c1713c WatchSource:0}: Error finding container 8ea96066e5ec82e998b8e9c47eb2a3afdefa5cadfcc09b930dccf9f834c1713c: Status 404 returned error can't find the container with id 8ea96066e5ec82e998b8e9c47eb2a3afdefa5cadfcc09b930dccf9f834c1713c
Jul 29 01:12:28.995720 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.995689    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 01:07:27 +0000 UTC" deadline="2028-02-26 01:55:15.394207456 +0000 UTC"
Jul 29 01:12:28.995806 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:28.995723    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13848h42m46.398489101s"
Jul 29 01:12:28.996107 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:28.996066    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ed3c06_7825_4d83_9da3_43877fb9780f.slice/crio-cd572a078cd04bc066a86bd08ae7ec44aea80541b1bb51b01ff8dfcfea98eeb1 WatchSource:0}: Error finding container cd572a078cd04bc066a86bd08ae7ec44aea80541b1bb51b01ff8dfcfea98eeb1: Status 404 returned error can't find the container with id cd572a078cd04bc066a86bd08ae7ec44aea80541b1bb51b01ff8dfcfea98eeb1
Jul 29 01:12:28.999486 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:28.999467    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd96a982_23af_4213_8fd8_27f14ef2cb2c.slice/crio-6af586ac0d3d3c065f8742486618feb9b34bc6f19f28de62b2689b04a9f11f6d WatchSource:0}: Error finding container 6af586ac0d3d3c065f8742486618feb9b34bc6f19f28de62b2689b04a9f11f6d: Status 404 returned error can't find the container with id 6af586ac0d3d3c065f8742486618feb9b34bc6f19f28de62b2689b04a9f11f6d
Jul 29 01:12:29.000525 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:29.000502    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530ed9b3_f52e_448f_a9a5_3c92dfb00ff6.slice/crio-34fbc3ef56d2d48afef11a3af9f35ccac8750223e2b3021f66a32fd2ce089786 WatchSource:0}: Error finding container 34fbc3ef56d2d48afef11a3af9f35ccac8750223e2b3021f66a32fd2ce089786: Status 404 returned error can't find the container with id 34fbc3ef56d2d48afef11a3af9f35ccac8750223e2b3021f66a32fd2ce089786
Jul 29 01:12:29.002024 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:12:29.001827    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6741d76d_6d73_4dad_b5ab_2cb9f3525cfd.slice/crio-5d36db39dcc96dab959194df72dc7e1661299e320df44f5f81aeffd4b6cbe3cf WatchSource:0}: Error finding container 5d36db39dcc96dab959194df72dc7e1661299e320df44f5f81aeffd4b6cbe3cf: Status 404 returned error can't find the container with id 5d36db39dcc96dab959194df72dc7e1661299e320df44f5f81aeffd4b6cbe3cf
Jul 29 01:12:29.118079 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.117953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qsvht" event={"ID":"9447e1e0-a540-4af2-b582-6628c24a78b6","Type":"ContainerStarted","Data":"2041522651a8ca3355210c61b605c1656adf3aee0a37743c102f39b46dd30407"}
Jul 29 01:12:29.119127 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.119102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979" event={"ID":"28e471cd-bab7-487b-9195-cb4e51d69100","Type":"ContainerStarted","Data":"4cb371f050f2bdb55fa6999e53fe9cf78aa32daeb6264e926ebb1367a0eba129"}
Jul 29 01:12:29.120104 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.120072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqr8t" event={"ID":"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a","Type":"ContainerStarted","Data":"c732533233ccb67c6d267b913e37d27b938966b35d8df439bf92d1c8c2c51114"}
Jul 29 01:12:29.121276 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.121242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerStarted","Data":"5d36db39dcc96dab959194df72dc7e1661299e320df44f5f81aeffd4b6cbe3cf"}
Jul 29 01:12:29.122258 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.122238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6kth6" event={"ID":"986c3fc6-5c33-494b-8e89-0483e1ddefc7","Type":"ContainerStarted","Data":"13ec2d96b6423e659ea4502f899e94f928c0b8f13f465619ecd9f8dec6c75e41"}
Jul 29 01:12:29.123398 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.123379    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8qrf" event={"ID":"fd96a982-23af-4213-8fd8-27f14ef2cb2c","Type":"ContainerStarted","Data":"6af586ac0d3d3c065f8742486618feb9b34bc6f19f28de62b2689b04a9f11f6d"}
Jul 29 01:12:29.124520 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.124500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2l955" event={"ID":"31ed3c06-7825-4d83-9da3-43877fb9780f","Type":"ContainerStarted","Data":"cd572a078cd04bc066a86bd08ae7ec44aea80541b1bb51b01ff8dfcfea98eeb1"}
Jul 29 01:12:29.126447 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.126427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal" event={"ID":"3616ae90a98efffd0c217d8bdda2d07e","Type":"ContainerStarted","Data":"91758f553025bb2d4b11be8c7a90ea005d42785138b46e2148074c764d236ced"}
Jul 29 01:12:29.127587 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.127569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"34fbc3ef56d2d48afef11a3af9f35ccac8750223e2b3021f66a32fd2ce089786"}
Jul 29 01:12:29.128876 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.128855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6lrj" event={"ID":"4686ca3b-0f27-45c9-92bc-c36426580d85","Type":"ContainerStarted","Data":"8ea96066e5ec82e998b8e9c47eb2a3afdefa5cadfcc09b930dccf9f834c1713c"}
Jul 29 01:12:29.143267 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.143223    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-49.ec2.internal" podStartSLOduration=2.14320691 podStartE2EDuration="2.14320691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:29.142681248 +0000 UTC m=+3.574563729" watchObservedRunningTime="2026-07-29 01:12:29.14320691 +0000 UTC m=+3.575089392"
Jul 29 01:12:29.227527 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.227500    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 01:12:29.586753 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.586717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:29.586947 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.586795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:29.586947 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.586939    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:29.587074 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.587013    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:31.586983936 +0000 UTC m=+6.018866397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:29.587465 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.587441    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:29.587465 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.587464    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:29.587605 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.587475    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:29.587605 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.587602    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:31.587506914 +0000 UTC m=+6.019389375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:29.688050 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.687509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:29.688050 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.687658    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:29.688050 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:29.687729    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:31.687707686 +0000 UTC m=+6.119590147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:29.996283 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.996134    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 01:07:27 +0000 UTC" deadline="2028-03-23 23:31:43.275410985 +0000 UTC"
Jul 29 01:12:29.996283 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:29.996207    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14494h19m13.279208078s"
Jul 29 01:12:30.113429 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:30.113397    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:30.113611 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:30.113537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:30.113940 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:30.113919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:30.114038 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:30.114019    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:30.114099 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:30.114081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:30.114187 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:30.114166    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:30.144561 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:30.144532    2564 generic.go:356] "Generic (PLEG): container finished" podID="2184daae0660a411ecef5fddf1a47bfc" containerID="03f1bd7e42fca22b5fa703b47cb425e9459a21cc5494daef4f795086cbde1283" exitCode=0
Jul 29 01:12:30.145482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:30.145457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal" event={"ID":"2184daae0660a411ecef5fddf1a47bfc","Type":"ContainerDied","Data":"03f1bd7e42fca22b5fa703b47cb425e9459a21cc5494daef4f795086cbde1283"}
Jul 29 01:12:31.166411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:31.166376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal" event={"ID":"2184daae0660a411ecef5fddf1a47bfc","Type":"ContainerStarted","Data":"98970b9e5e0a4abb72f2cb82883a4ed6d24738023e56fadb5f9afa5f8e6580a3"}
Jul 29 01:12:31.606056 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:31.606017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:31.606232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:31.606101    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:31.606317 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606280    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:31.606317 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606301    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:31.606317 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606314    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:31.606466 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606375    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:35.606355341 +0000 UTC m=+10.038237818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:31.606804 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606785    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:31.606873 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.606840    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:35.606824552 +0000 UTC m=+10.038707013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:31.706845 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:31.706810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:31.707019 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.707004    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:31.707099 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:31.707068    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:35.707049671 +0000 UTC m=+10.138932129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:32.111905 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:32.111836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:32.112071 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:32.111836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:32.112071 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:32.111959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:32.112071 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:32.112060    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:32.112250 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:32.112119    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:32.112250 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:32.112202    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:34.110989 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:34.110914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:34.111481 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:34.111052    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:34.111481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:34.110914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:34.111593 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:34.111525    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:34.111773 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:34.111741    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:34.111883 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:34.111866    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:35.640817 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:35.640783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:35.640862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.640995    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.641015    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.641035    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.641046    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.641058    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:43.641039768 +0000 UTC m=+18.072922246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:35.641307 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.641081    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:43.641071329 +0000 UTC m=+18.072953789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:35.741950 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:35.741388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:35.741950 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.741587    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:35.741950 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:35.741644    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:43.741626751 +0000 UTC m=+18.173509213 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:36.111673    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:36.111779    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:36.111804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:36.111878    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:36.111984    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:36.112139 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:36.112086    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:38.110356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:38.110388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:38.110356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:38.110473    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:38.110569    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:38.110711 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:38.110630    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:40.110619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:40.110578    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:40.111060 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:40.110589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:40.111060 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:40.110709    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:40.111060 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:40.110681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:40.111060 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:40.110794    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:40.111060 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:40.110903    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:42.110848 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:42.110783    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:42.110848 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:42.110782    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:42.111389 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:42.110921    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:42.111389 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:42.110959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:42.111389 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:42.111031    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:42.111389 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:42.111125    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:43.700808 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:43.700774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:43.700842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.700923    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.700985    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:59.700967102 +0000 UTC m=+34.132849581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.700988    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.701007    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.701020    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:43.701255 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.701066    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:12:59.701050605 +0000 UTC m=+34.132933066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:43.802191 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:43.802159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:43.802349 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.802269    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:43.802349 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:43.802339    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:12:59.802319883 +0000 UTC m=+34.234202357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:44.110981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:44.110946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:44.110981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:44.110967    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:44.111218 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:44.110951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:44.111218 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:44.111088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:44.111218 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:44.111177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:44.111353 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:44.111266    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:46.111482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:46.111454    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:46.111903 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:46.111563    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:46.111903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:46.111597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:46.111903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:46.111618    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:46.111903 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:46.111705    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:46.111903 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:46.111790    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:48.111463 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.111244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:48.112143 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.111244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:48.112143 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:48.111495    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:48.112143 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:48.111587    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:48.112143 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.111243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:48.112143 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:48.111685    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:48.198547 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.198513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979" event={"ID":"28e471cd-bab7-487b-9195-cb4e51d69100","Type":"ContainerStarted","Data":"212633724a9dee1d211d33cc5534299cd331e5d854ce8cc2f8c0d1a7c6be2d7b"}
Jul 29 01:12:48.200073 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.200021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tqr8t" event={"ID":"1ea819b9-2b2d-4f81-aee1-d6e3636ee41a","Type":"ContainerStarted","Data":"a673b2b2397f0cd736b6b5bdbb8683de64245ca769b8c4af82d71b9b4b3d050d"}
Jul 29 01:12:48.201726 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.201699    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="4fa308446d4b44f983103cffa23ce5214cb00a4ad39eae0ede8e1d930d4f43fd" exitCode=0
Jul 29 01:12:48.201819 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.201778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"4fa308446d4b44f983103cffa23ce5214cb00a4ad39eae0ede8e1d930d4f43fd"}
Jul 29 01:12:48.203619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.203596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6kth6" event={"ID":"986c3fc6-5c33-494b-8e89-0483e1ddefc7","Type":"ContainerStarted","Data":"ed03d4b98fc1dbaf6e155f2115f16cc46dbdf4e9df6b972de790f001cd638446"}
Jul 29 01:12:48.205478 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.205436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2l955" event={"ID":"31ed3c06-7825-4d83-9da3-43877fb9780f","Type":"ContainerStarted","Data":"4b7cbccd40bc65bcfa2fedeae0c7484f8b1965ee6dd96a4ca865e1749ac119e1"}
Jul 29 01:12:48.208595 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"8d502834710974fcdf19f8bc7e027e7492dff201475e160e94f8d40ce2111ae3"}
Jul 29 01:12:48.208690 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"cdccc0231c48a5314a3f0c32001000ae1e763d62807e34f8a5f86b916d392002"}
Jul 29 01:12:48.208690 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"47c0237a5f2daa9f49d289902afbc6fb34913162a3f99060b2b8a7a918639b14"}
Jul 29 01:12:48.208793 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"00c0ead821138f64a7c9321ee0a53ca3a4a973cf32db9a809faa2e4c1fd29e3e"}
Jul 29 01:12:48.208793 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"b86142e7a77b734acc0f08f6d865f41a98e26d7a2d6de8a6d60bce8d22bfb6c9"}
Jul 29 01:12:48.208896 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.208813    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"881089327a3aab0ca5046b5312088855a171a1d2eb99e9ffb50871e09058a611"}
Jul 29 01:12:48.210020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.209994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6lrj" event={"ID":"4686ca3b-0f27-45c9-92bc-c36426580d85","Type":"ContainerStarted","Data":"223531ebb5a4467a226fe19bbb56dcf7e1a505660a60ba167eb2a0fc44608a7e"}
Jul 29 01:12:48.211497 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.211474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qsvht" event={"ID":"9447e1e0-a540-4af2-b582-6628c24a78b6","Type":"ContainerStarted","Data":"b4e25b009678d1cf8a6d2279249a246a2afa742c1a8dec4c6b7a8f2c9673ec3a"}
Jul 29 01:12:48.216627 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.216584    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-49.ec2.internal" podStartSLOduration=21.216569086 podStartE2EDuration="21.216569086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:31.182733518 +0000 UTC m=+5.614615998" watchObservedRunningTime="2026-07-29 01:12:48.216569086 +0000 UTC m=+22.648451565"
Jul 29 01:12:48.216897 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.216872    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tqr8t" podStartSLOduration=3.871160818 podStartE2EDuration="22.216864087s" totalImagesPullingTime="18.345703269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:48.216655199 +0000 UTC m=+22.648537681" watchObservedRunningTime="2026-07-29 01:12:48.216864087 +0000 UTC m=+22.648746566"
Jul 29 01:12:48.235367 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.235330    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v6lrj" podStartSLOduration=3.824916965 podStartE2EDuration="22.235316271s" totalImagesPullingTime="18.410399306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:48.234707607 +0000 UTC m=+22.666590087" watchObservedRunningTime="2026-07-29 01:12:48.235316271 +0000 UTC m=+22.667198752"
Jul 29 01:12:48.251499 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.251458    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6kth6" podStartSLOduration=11.91099619 podStartE2EDuration="22.251442963s" totalImagesPullingTime="10.340446773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:48.25121989 +0000 UTC m=+22.683102371" watchObservedRunningTime="2026-07-29 01:12:48.251442963 +0000 UTC m=+22.683325443"
Jul 29 01:12:48.287520 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.287486    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2l955" podStartSLOduration=3.931285879 podStartE2EDuration="22.287474497s" totalImagesPullingTime="18.356188618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:48.287416479 +0000 UTC m=+22.719298953" watchObservedRunningTime="2026-07-29 01:12:48.287474497 +0000 UTC m=+22.719356976"
Jul 29 01:12:48.306495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.306459    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qsvht" podStartSLOduration=3.965365132 podStartE2EDuration="22.306449783s" totalImagesPullingTime="18.341084651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:48.306087726 +0000 UTC m=+22.737970207" watchObservedRunningTime="2026-07-29 01:12:48.306449783 +0000 UTC m=+22.738332263"
Jul 29 01:12:48.734133 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:48.733960    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 01:12:49.034600 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.034137    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T01:12:48.73397628Z","UUID":"4853b696-6f1a-4bf7-bf3b-d5b70ffb6269","Handler":null,"Name":"","Endpoint":""}
Jul 29 01:12:49.035560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.035538    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 01:12:49.035676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.035583    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 01:12:49.216068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.216035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979" event={"ID":"28e471cd-bab7-487b-9195-cb4e51d69100","Type":"ContainerStarted","Data":"ff4aff97532d72f815afde1b0ac84097cd25355d97fffc3dc8b0e477fd2c187d"}
Jul 29 01:12:49.218066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.218012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8qrf" event={"ID":"fd96a982-23af-4213-8fd8-27f14ef2cb2c","Type":"ContainerStarted","Data":"1f032bf4cc7d84bc7c5ec41a176caece27665fb4e5cd0e20f7ad7052964ba487"}
Jul 29 01:12:49.234397 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:49.234359    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h8qrf" podStartSLOduration=4.88216658 podStartE2EDuration="23.234345028s" totalImagesPullingTime="18.352178448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:49.234022616 +0000 UTC m=+23.665905097" watchObservedRunningTime="2026-07-29 01:12:49.234345028 +0000 UTC m=+23.666227509"
Jul 29 01:12:50.110900 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.110869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:50.111089 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.110869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:50.111089 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.111023    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:50.111089 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:50.111015    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:50.111319 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:50.111118    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:50.111319 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:50.111218    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:50.222290 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.222244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979" event={"ID":"28e471cd-bab7-487b-9195-cb4e51d69100","Type":"ContainerStarted","Data":"a451310ee98c7b7a33d704725a192673dbc60779b348d3c5c937dc8e74e56015"}
Jul 29 01:12:50.228959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.227713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"be15758d96b1870a465cd1f69b9b738966d65a3b926968cc13b0575f6393415a"}
Jul 29 01:12:50.240835 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:50.240762    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn979" podStartSLOduration=3.876628883 podStartE2EDuration="24.24074273s" totalImagesPullingTime="20.364113847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:50.240007334 +0000 UTC m=+24.671889815" watchObservedRunningTime="2026-07-29 01:12:50.24074273 +0000 UTC m=+24.672625256"
Jul 29 01:12:51.626008 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:51.625811    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:51.626536 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:51.626521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:52.110423 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:52.110389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:52.110605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:52.110389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:52.110605 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:52.110528    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:52.110605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:52.110538    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:52.110767 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:52.110605    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:52.110767 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:52.110678    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:52.232025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:52.232005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:52.232532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:52.232517    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6kth6"
Jul 29 01:12:53.235221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:53.235006    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="f6e09a9ca8affcb5f5e5098d85051972195e1cbd21436f5f1a46d9e4b879342c" exitCode=0
Jul 29 01:12:53.235978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:53.235093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"f6e09a9ca8affcb5f5e5098d85051972195e1cbd21436f5f1a46d9e4b879342c"}
Jul 29 01:12:53.238966 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:53.238933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" event={"ID":"530ed9b3-f52e-448f-a9a5-3c92dfb00ff6","Type":"ContainerStarted","Data":"adb00c9a1b6cceb23794647fcd178c5366f05e45b4d6e065f8cff33bd3ea902b"}
Jul 29 01:12:53.287242 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:53.287187    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z" podStartSLOduration=8.873678813 podStartE2EDuration="27.28716854s" totalImagesPullingTime="18.413489727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:53.284494335 +0000 UTC m=+27.716376796" watchObservedRunningTime="2026-07-29 01:12:53.28716854 +0000 UTC m=+27.719051021"
Jul 29 01:12:54.110528 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.110455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:54.110528 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.110478    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:54.110528 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.110514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:54.110745 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.110609    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:54.110745 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.110682    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:54.110808 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.110774    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:54.241736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.241693    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:54.242176 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.241792    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:54.242176 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.241840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:54.257418 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.257391    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:54.257602 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.257494    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:12:54.481953 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.481918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tb8c"]
Jul 29 01:12:54.482102 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.482019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:54.482166 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.482113    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:54.484526 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.484495    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qw64x"]
Jul 29 01:12:54.484636 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.484596    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:54.484716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.484692    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:54.485194 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.485168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sh9cm"]
Jul 29 01:12:54.485292 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:54.485250    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:54.485378 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:54.485347    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:55.245530 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:55.245496    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="7e2c5dbef7adeb96c7eaab94497d6151060f75e0d34fa33773ac8a03fc15d629" exitCode=0
Jul 29 01:12:55.245943 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:55.245581    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"7e2c5dbef7adeb96c7eaab94497d6151060f75e0d34fa33773ac8a03fc15d629"}
Jul 29 01:12:56.111774 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:56.111748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:56.111946 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:56.111831    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:56.112026 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:56.111994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:56.112181 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:56.112143    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:56.112181 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:56.112176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:56.112298 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:56.112267    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:57.252328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:57.252079    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="a3be0f73e69a1567b1159f8751476705b21659d294b9d9ce7349093e8909a516" exitCode=0
Jul 29 01:12:57.252328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:57.252186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"a3be0f73e69a1567b1159f8751476705b21659d294b9d9ce7349093e8909a516"}
Jul 29 01:12:58.110501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:58.110468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:58.110660 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:58.110580    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:12:58.110660 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:58.110471    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:58.110744 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:58.110468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:58.110744 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:58.110674    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:12:58.110810 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:58.110780    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:12:59.729111 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:59.729069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:59.729166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729243    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729315    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs podName:e29ae934-2ad2-4b67-8d4c-27ca1d30a571 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:31.729294112 +0000 UTC m=+66.161176571 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs") pod "network-metrics-daemon-2tb8c" (UID: "e29ae934-2ad2-4b67-8d4c-27ca1d30a571") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729320    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729336    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729350    2564 projected.go:196] Error preparing data for projected volume kube-api-access-h58zj for pod openshift-network-diagnostics/network-check-target-qw64x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:59.729716 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.729401    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj podName:7e9a5312-259b-45aa-baac-3ddd9425f781 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:31.729385861 +0000 UTC m=+66.161268321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h58zj" (UniqueName: "kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj") pod "network-check-target-qw64x" (UID: "7e9a5312-259b-45aa-baac-3ddd9425f781") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 01:12:59.829797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:12:59.829760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:12:59.829980 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.829904    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 01:12:59.829980 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:12:59.829973    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret podName:9438ab0c-aec4-41b1-b091-e541c119b72c nodeName:}" failed. No retries permitted until 2026-07-29 01:13:31.829953494 +0000 UTC m=+66.261835965 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret") pod "global-pull-secret-syncer-sh9cm" (UID: "9438ab0c-aec4-41b1-b091-e541c119b72c") : object "kube-system"/"original-pull-secret" not registered
Jul 29 01:13:00.110886 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.110825    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:13:00.111073 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.110834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:13:00.111073 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.110959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sh9cm" podUID="9438ab0c-aec4-41b1-b091-e541c119b72c"
Jul 29 01:13:00.111225 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.111076    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tb8c" podUID="e29ae934-2ad2-4b67-8d4c-27ca1d30a571"
Jul 29 01:13:00.111225 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.111101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:00.111225 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.111216    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw64x" podUID="7e9a5312-259b-45aa-baac-3ddd9425f781"
Jul 29 01:13:00.369960 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.369885    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-49.ec2.internal" event="NodeReady"
Jul 29 01:13:00.370121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.370017    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 01:13:00.408256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.408223    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:13:00.410631 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.410614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.414281 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.414259    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 01:13:00.414414 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.414356    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sgklk\""
Jul 29 01:13:00.414677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.414656    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 01:13:00.414901 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.414885    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 01:13:00.423496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.423472    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 01:13:00.427266 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.427243    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:13:00.431450 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.431431    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"]
Jul 29 01:13:00.433932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.433909    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"]
Jul 29 01:13:00.434066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.434046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.435539 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.435523    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"]
Jul 29 01:13:00.435671 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.435653    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"
Jul 29 01:13:00.437373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.437356    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wkb2l"]
Jul 29 01:13:00.437497 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.437480    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.439211 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.439192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.439719 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.439694    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.441127 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441106    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.441230 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441203    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.441268 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-v57kw\""
Jul 29 01:13:00.441356 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441336    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 01:13:00.441421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441409    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 01:13:00.441493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vbv74\""
Jul 29 01:13:00.441493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441468    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.441646 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441611    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.441858 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441804    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 01:13:00.441858 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441850    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.442005 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.441989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mr7n9\""
Jul 29 01:13:00.442088 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 01:13:00.442353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442335    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.442442 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442360    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 01:13:00.442442 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442341    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 01:13:00.442442 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442331    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.442872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.442858    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fzdk5\""
Jul 29 01:13:00.450198 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.450178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 01:13:00.452566 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.452318    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"]
Jul 29 01:13:00.453518 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.453496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"]
Jul 29 01:13:00.454475 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.454443    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"]
Jul 29 01:13:00.455118 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.455085    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lz9dv"]
Jul 29 01:13:00.457740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.457724    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kt2c4"]
Jul 29 01:13:00.458835 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.457928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.460028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.460006    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-45szv"]
Jul 29 01:13:00.460190 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.460142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.461191 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.461171    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 01:13:00.461293 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.461218    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 01:13:00.462923 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.462902    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.463141 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.463121    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 01:13:00.463410 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.463385    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6gwkb\""
Jul 29 01:13:00.463527 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.463509    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n8mzz\""
Jul 29 01:13:00.463714 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.463697    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.464965 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.464946    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 01:13:00.470366 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.470347    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"]
Jul 29 01:13:00.471525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.470451    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.477491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.477466    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"]
Jul 29 01:13:00.478581 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.478564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.479257 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.479236    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 01:13:00.479394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.479237    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.479498 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.479303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.479498 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.479348    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zbhk\""
Jul 29 01:13:00.479624 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.479610    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 01:13:00.480071 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.480055    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"]
Jul 29 01:13:00.480944 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.480928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.482320 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.482303    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"]
Jul 29 01:13:00.482607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.482588    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.483091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.483391 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483372    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.483492 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483329    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 01:13:00.483552 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483362    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 01:13:00.483552 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483215    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gmgrx\""
Jul 29 01:13:00.484006 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.483990    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.484108 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484091    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xbshl\""
Jul 29 01:13:00.484204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484183    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.484368 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484297    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 01:13:00.484935 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484918    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 01:13:00.485001 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"
Jul 29 01:13:00.485110 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.484923    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-857b7c775-7mj7f"]
Jul 29 01:13:00.486118 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.486100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 01:13:00.486812 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.486795    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fxl8k\""
Jul 29 01:13:00.487419 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487395    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"]
Jul 29 01:13:00.487585 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-45szv"]
Jul 29 01:13:00.487676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"]
Jul 29 01:13:00.487676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487526    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.487676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"]
Jul 29 01:13:00.487676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kt2c4"]
Jul 29 01:13:00.487676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lz9dv"]
Jul 29 01:13:00.487920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487693    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wkb2l"]
Jul 29 01:13:00.487920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.487715    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"]
Jul 29 01:13:00.488179 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.488137    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h58cr\""
Jul 29 01:13:00.488762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.488745    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.488975 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.488950    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.490235 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.490213    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-857b7c775-7mj7f"]
Jul 29 01:13:00.490631 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.490615    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 01:13:00.491054 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491025    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 01:13:00.491466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491445    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 01:13:00.491545 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491476    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 01:13:00.491716 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rd2vs\""
Jul 29 01:13:00.491824 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491720    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 01:13:00.491916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.491731    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 01:13:00.535095 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535263 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppnrl\" (UniqueName: \"kubernetes.io/projected/241109d5-1076-4d7c-bedb-0a1b70487d17-kube-api-access-ppnrl\") pod \"volume-data-source-validator-549c6d54cf-djlvl\" (UID: \"241109d5-1076-4d7c-bedb-0a1b70487d17\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"
Jul 29 01:13:00.535263 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535369 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535369 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qclnk\" (UniqueName: \"kubernetes.io/projected/198bd2b4-3160-4256-a39f-db5f57868a3f-kube-api-access-qclnk\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.535369 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535342    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8q4f\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.535495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dab08fa-1154-4caa-80f3-ecfd50ee4351-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.535674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/198bd2b4-3160-4256-a39f-db5f57868a3f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.535674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.535794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535671    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dab08fa-1154-4caa-80f3-ecfd50ee4351-config\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.535794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.535700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs22g\" (UniqueName: \"kubernetes.io/projected/3dab08fa-1154-4caa-80f3-ecfd50ee4351-kube-api-access-zs22g\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.636427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.636427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636389    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4k2d\" (UniqueName: \"kubernetes.io/projected/711d634c-0515-4f70-9f82-6f387aeb020a-kube-api-access-q4k2d\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.636427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-config\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04414c82-3152-4607-ae22-cae517c84cd4-cert\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-tmp\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-snapshots\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-default-certificate\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.636676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppnrl\" (UniqueName: \"kubernetes.io/projected/241109d5-1076-4d7c-bedb-0a1b70487d17-kube-api-access-ppnrl\") pod \"volume-data-source-validator-549c6d54cf-djlvl\" (UID: \"241109d5-1076-4d7c-bedb-0a1b70487d17\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82d56\" (UniqueName: \"kubernetes.io/projected/49d40c02-c488-48a4-820d-257a0d4ee7b3-kube-api-access-82d56\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qclnk\" (UniqueName: \"kubernetes.io/projected/198bd2b4-3160-4256-a39f-db5f57868a3f-kube-api-access-qclnk\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-serving-cert\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711d634c-0515-4f70-9f82-6f387aeb020a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k9s8\" (UniqueName: \"kubernetes.io/projected/53c70359-d905-4b35-a16f-6dd4d4443f68-kube-api-access-6k9s8\") pod \"network-check-source-7d88cb55d8-d659t\" (UID: \"53c70359-d905-4b35-a16f-6dd4d4443f68\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"
Jul 29 01:13:00.636945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.636946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b3711a-bb50-4704-b92e-1bd02d314cd2-config-volume\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dab08fa-1154-4caa-80f3-ecfd50ee4351-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d35aad0-55d6-4d74-a7d4-fb149201c916-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/198bd2b4-3160-4256-a39f-db5f57868a3f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.637120    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-trusted-ca\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637172    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.637207    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.137185892 +0000 UTC m=+35.569068370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dab08fa-1154-4caa-80f3-ecfd50ee4351-config\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs22g\" (UniqueName: \"kubernetes.io/projected/3dab08fa-1154-4caa-80f3-ecfd50ee4351-kube-api-access-zs22g\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fslh\" (UniqueName: \"kubernetes.io/projected/04414c82-3152-4607-ae22-cae517c84cd4-kube-api-access-8fslh\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.637339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltk4w\" (UniqueName: \"kubernetes.io/projected/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-kube-api-access-ltk4w\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkfw2\" (UniqueName: \"kubernetes.io/projected/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-kube-api-access-bkfw2\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8rn9\" (UniqueName: \"kubernetes.io/projected/870dc954-5b03-4306-859f-484b01bb4701-kube-api-access-b8rn9\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.637162    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.637491    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-stats-auth\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.637535    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.137520366 +0000 UTC m=+35.569402844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b3711a-bb50-4704-b92e-1bd02d314cd2-tmp-dir\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49d40c02-c488-48a4-820d-257a0d4ee7b3-serving-cert\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.637992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dab08fa-1154-4caa-80f3-ecfd50ee4351-config\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637926    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4r7d\" (UniqueName: \"kubernetes.io/projected/95b3711a-bb50-4704-b92e-1bd02d314cd2-kube-api-access-l4r7d\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.637994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8q4f\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.638023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711d634c-0515-4f70-9f82-6f387aeb020a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.638215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/198bd2b4-3160-4256-a39f-db5f57868a3f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.638686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.638393    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.642095 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.642067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dab08fa-1154-4caa-80f3-ecfd50ee4351-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.642232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.642203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.642232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.642210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.646100 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.646073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qclnk\" (UniqueName: \"kubernetes.io/projected/198bd2b4-3160-4256-a39f-db5f57868a3f-kube-api-access-qclnk\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:00.646342 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.646298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs22g\" (UniqueName: \"kubernetes.io/projected/3dab08fa-1154-4caa-80f3-ecfd50ee4351-kube-api-access-zs22g\") pod \"service-ca-operator-5d6c75f84f-gjw2v\" (UID: \"3dab08fa-1154-4caa-80f3-ecfd50ee4351\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.646575 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.646554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.646776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.646754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8q4f\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:00.648378 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.648355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppnrl\" (UniqueName: \"kubernetes.io/projected/241109d5-1076-4d7c-bedb-0a1b70487d17-kube-api-access-ppnrl\") pod \"volume-data-source-validator-549c6d54cf-djlvl\" (UID: \"241109d5-1076-4d7c-bedb-0a1b70487d17\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"
Jul 29 01:13:00.739254 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-tmp\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-snapshots\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-default-certificate\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.739484    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.739542    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.239524081 +0000 UTC m=+35.671406540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82d56\" (UniqueName: \"kubernetes.io/projected/49d40c02-c488-48a4-820d-257a0d4ee7b3-kube-api-access-82d56\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-serving-cert\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711d634c-0515-4f70-9f82-6f387aeb020a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k9s8\" (UniqueName: \"kubernetes.io/projected/53c70359-d905-4b35-a16f-6dd4d4443f68-kube-api-access-6k9s8\") pod \"network-check-source-7d88cb55d8-d659t\" (UID: \"53c70359-d905-4b35-a16f-6dd4d4443f68\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b3711a-bb50-4704-b92e-1bd02d314cd2-config-volume\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.739999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-snapshots\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d35aad0-55d6-4d74-a7d4-fb149201c916-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-trusted-ca\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740174    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fslh\" (UniqueName: \"kubernetes.io/projected/04414c82-3152-4607-ae22-cae517c84cd4-kube-api-access-8fslh\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltk4w\" (UniqueName: \"kubernetes.io/projected/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-kube-api-access-ltk4w\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740253    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkfw2\" (UniqueName: \"kubernetes.io/projected/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-kube-api-access-bkfw2\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.740332    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.740398    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.240380835 +0000 UTC m=+35.672263313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49d40c02-c488-48a4-820d-257a0d4ee7b3-tmp\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49d40c02-c488-48a4-820d-257a0d4ee7b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/711d634c-0515-4f70-9f82-6f387aeb020a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.740762 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740718    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8rn9\" (UniqueName: \"kubernetes.io/projected/870dc954-5b03-4306-859f-484b01bb4701-kube-api-access-b8rn9\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-stats-auth\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b3711a-bb50-4704-b92e-1bd02d314cd2-tmp-dir\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.740893    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49d40c02-c488-48a4-820d-257a0d4ee7b3-serving-cert\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.740959    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.240944367 +0000 UTC m=+35.672826838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.740992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4r7d\" (UniqueName: \"kubernetes.io/projected/95b3711a-bb50-4704-b92e-1bd02d314cd2-kube-api-access-l4r7d\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711d634c-0515-4f70-9f82-6f387aeb020a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4k2d\" (UniqueName: \"kubernetes.io/projected/711d634c-0515-4f70-9f82-6f387aeb020a-kube-api-access-q4k2d\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-config\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.741140    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.241117797 +0000 UTC m=+35.673000258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04414c82-3152-4607-ae22-cae517c84cd4-cert\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.741345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741191    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b3711a-bb50-4704-b92e-1bd02d314cd2-config-volume\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.742030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-trusted-ca\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.742030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3d35aad0-55d6-4d74-a7d4-fb149201c916-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:00.742030 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.741528    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:00.742030 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:00.741572    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:01.241557933 +0000 UTC m=+35.673440399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:00.742030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.741838    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-config\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.742298 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.742100    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b3711a-bb50-4704-b92e-1bd02d314cd2-tmp-dir\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.742706 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.742664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-default-certificate\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.744133 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.744110    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-stats-auth\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.744249 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.744180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49d40c02-c488-48a4-820d-257a0d4ee7b3-serving-cert\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.744249 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.744185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-serving-cert\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.744341 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.744250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04414c82-3152-4607-ae22-cae517c84cd4-cert\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.744434 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.744412    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/711d634c-0515-4f70-9f82-6f387aeb020a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.749046 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.749022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkfw2\" (UniqueName: \"kubernetes.io/projected/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-kube-api-access-bkfw2\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:00.749988 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.749945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82d56\" (UniqueName: \"kubernetes.io/projected/49d40c02-c488-48a4-820d-257a0d4ee7b3-kube-api-access-82d56\") pod \"insights-operator-5885fdb759-wkb2l\" (UID: \"49d40c02-c488-48a4-820d-257a0d4ee7b3\") " pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.752165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.752110    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4r7d\" (UniqueName: \"kubernetes.io/projected/95b3711a-bb50-4704-b92e-1bd02d314cd2-kube-api-access-l4r7d\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:00.752264 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.752208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fslh\" (UniqueName: \"kubernetes.io/projected/04414c82-3152-4607-ae22-cae517c84cd4-kube-api-access-8fslh\") pod \"ingress-canary-45szv\" (UID: \"04414c82-3152-4607-ae22-cae517c84cd4\") " pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.752482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.752431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8rn9\" (UniqueName: \"kubernetes.io/projected/870dc954-5b03-4306-859f-484b01bb4701-kube-api-access-b8rn9\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:00.752894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.752868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4k2d\" (UniqueName: \"kubernetes.io/projected/711d634c-0515-4f70-9f82-6f387aeb020a-kube-api-access-q4k2d\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl\" (UID: \"711d634c-0515-4f70-9f82-6f387aeb020a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.752995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.752969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k9s8\" (UniqueName: \"kubernetes.io/projected/53c70359-d905-4b35-a16f-6dd4d4443f68-kube-api-access-6k9s8\") pod \"network-check-source-7d88cb55d8-d659t\" (UID: \"53c70359-d905-4b35-a16f-6dd4d4443f68\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"
Jul 29 01:13:00.753853 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.753837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltk4w\" (UniqueName: \"kubernetes.io/projected/eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4-kube-api-access-ltk4w\") pod \"console-operator-584dcb6b54-kt2c4\" (UID: \"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4\") " pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.754565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.754548    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"
Jul 29 01:13:00.762955 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.762937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"
Jul 29 01:13:00.780670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.780647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wkb2l"
Jul 29 01:13:00.797602 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.797580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:00.805246 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.805230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-45szv"
Jul 29 01:13:00.811942 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.811924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"
Jul 29 01:13:00.831667 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:00.831637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"
Jul 29 01:13:01.144689 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.144656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:01.144689 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.144694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:01.144909 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.144830    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:01.144978 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.144922    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.144902092 +0000 UTC m=+36.576784567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:01.144978 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.144832    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:01.144978 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.144942    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:01.145108 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.144982    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.144971072 +0000 UTC m=+36.576853545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.245955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.246248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.246289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.246367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.246400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.246537    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.246597    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.246579171 +0000 UTC m=+36.678461644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247035    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247086    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.247070821 +0000 UTC m=+36.678953282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247164    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247196    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.247186138 +0000 UTC m=+36.679068610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247276    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.247265555 +0000 UTC m=+36.679148027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247331    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:01.247412 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:01.247360    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:02.247348711 +0000 UTC m=+36.679231183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:01.461481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.461413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-kt2c4"]
Jul 29 01:13:01.465445 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.465365    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t"]
Jul 29 01:13:01.466077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.465911    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl"]
Jul 29 01:13:01.476986 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.476959    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v"]
Jul 29 01:13:01.480287 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.480264    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wkb2l"]
Jul 29 01:13:01.488297 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.488277    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl"]
Jul 29 01:13:01.490991 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:01.490966    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-45szv"]
Jul 29 01:13:01.494549 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:01.494527    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241109d5_1076_4d7c_bedb_0a1b70487d17.slice/crio-3f34a0740232672a92f819667fa4eef0064329083f46763f2691b29c9ed03b6b WatchSource:0}: Error finding container 3f34a0740232672a92f819667fa4eef0064329083f46763f2691b29c9ed03b6b: Status 404 returned error can't find the container with id 3f34a0740232672a92f819667fa4eef0064329083f46763f2691b29c9ed03b6b
Jul 29 01:13:01.495561 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:01.495539    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d40c02_c488_48a4_820d_257a0d4ee7b3.slice/crio-38c09ea22e153758e62d2deaf39b847d5b5246bca621d874df16192d1621c858 WatchSource:0}: Error finding container 38c09ea22e153758e62d2deaf39b847d5b5246bca621d874df16192d1621c858: Status 404 returned error can't find the container with id 38c09ea22e153758e62d2deaf39b847d5b5246bca621d874df16192d1621c858
Jul 29 01:13:01.496996 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:01.496788    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dab08fa_1154_4caa_80f3_ecfd50ee4351.slice/crio-027b9672b50869131e71378c6f0b1f9b18a6b59487d17f1ca7ee1c5d6fbd46af WatchSource:0}: Error finding container 027b9672b50869131e71378c6f0b1f9b18a6b59487d17f1ca7ee1c5d6fbd46af: Status 404 returned error can't find the container with id 027b9672b50869131e71378c6f0b1f9b18a6b59487d17f1ca7ee1c5d6fbd46af
Jul 29 01:13:01.498234 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:01.498208    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04414c82_3152_4607_ae22_cae517c84cd4.slice/crio-996891b8c232881709c91440c40d51d80798f03af7a92e14bbed434a3cf06f05 WatchSource:0}: Error finding container 996891b8c232881709c91440c40d51d80798f03af7a92e14bbed434a3cf06f05: Status 404 returned error can't find the container with id 996891b8c232881709c91440c40d51d80798f03af7a92e14bbed434a3cf06f05
Jul 29 01:13:01.506532 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:01.506510    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebd4f30_39fe_4cf3_abf6_d7fd1cc5cff4.slice/crio-0fc71800d02b0616838844af3a56f5baa28f5685c094e877ac55cafac81efb64 WatchSource:0}: Error finding container 0fc71800d02b0616838844af3a56f5baa28f5685c094e877ac55cafac81efb64: Status 404 returned error can't find the container with id 0fc71800d02b0616838844af3a56f5baa28f5685c094e877ac55cafac81efb64
Jul 29 01:13:02.112634 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.112597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:13:02.113420 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.113398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:13:02.113605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.113586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:02.117239 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.117218    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 01:13:02.117471 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.117455    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nbzfd\""
Jul 29 01:13:02.120135 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.119876    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 01:13:02.120135 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.119951    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82b2w\""
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.156603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.156644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.157270    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.157328    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.157310775 +0000 UTC m=+38.589193238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.157391    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.157401    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:02.157678 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.157426    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.157418097 +0000 UTC m=+38.589300555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.257728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.257785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.257817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.257888    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.257894    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.257919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.257979    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.257948772 +0000 UTC m=+38.689831248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258037    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258101    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.25808504 +0000 UTC m=+38.689967500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258177    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258209    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.258198506 +0000 UTC m=+38.690080968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258284    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.258273548 +0000 UTC m=+38.690156009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258340    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:02.258396 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:02.258369    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:04.258357489 +0000 UTC m=+38.690239949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:02.272816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.272727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t" event={"ID":"53c70359-d905-4b35-a16f-6dd4d4443f68","Type":"ContainerStarted","Data":"c2b31c5836d5ef5b7dd8d849c1fbddb1dc190fc82e6e1befb044cb068247267d"}
Jul 29 01:13:02.275416 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.275376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl" event={"ID":"711d634c-0515-4f70-9f82-6f387aeb020a","Type":"ContainerStarted","Data":"fa9a187cc8b857ba577dc81d437cf2ab75858e841f224431c3109be704782a6b"}
Jul 29 01:13:02.280033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.279974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wkb2l" event={"ID":"49d40c02-c488-48a4-820d-257a0d4ee7b3","Type":"ContainerStarted","Data":"38c09ea22e153758e62d2deaf39b847d5b5246bca621d874df16192d1621c858"}
Jul 29 01:13:02.284313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.284278    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="559ebe7564b498fc68a757c3b5924d3fd2dfb530188efd1baa25981ea2e5f508" exitCode=0
Jul 29 01:13:02.284417 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.284372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"559ebe7564b498fc68a757c3b5924d3fd2dfb530188efd1baa25981ea2e5f508"}
Jul 29 01:13:02.287237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.287175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-45szv" event={"ID":"04414c82-3152-4607-ae22-cae517c84cd4","Type":"ContainerStarted","Data":"996891b8c232881709c91440c40d51d80798f03af7a92e14bbed434a3cf06f05"}
Jul 29 01:13:02.289598 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.289542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" event={"ID":"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4","Type":"ContainerStarted","Data":"0fc71800d02b0616838844af3a56f5baa28f5685c094e877ac55cafac81efb64"}
Jul 29 01:13:02.292960 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.292900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v" event={"ID":"3dab08fa-1154-4caa-80f3-ecfd50ee4351","Type":"ContainerStarted","Data":"027b9672b50869131e71378c6f0b1f9b18a6b59487d17f1ca7ee1c5d6fbd46af"}
Jul 29 01:13:02.294515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:02.294461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl" event={"ID":"241109d5-1076-4d7c-bedb-0a1b70487d17","Type":"ContainerStarted","Data":"3f34a0740232672a92f819667fa4eef0064329083f46763f2691b29c9ed03b6b"}
Jul 29 01:13:03.304058 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:03.303732    2564 generic.go:356] "Generic (PLEG): container finished" podID="6741d76d-6d73-4dad-b5ab-2cb9f3525cfd" containerID="d8f65a035d37680a27eec1e4d732eae8633d3661e254e06311d2beb6ce353003" exitCode=0
Jul 29 01:13:03.304058 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:03.303994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerDied","Data":"d8f65a035d37680a27eec1e4d732eae8633d3661e254e06311d2beb6ce353003"}
Jul 29 01:13:04.180301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.180213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:04.180301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.180266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:04.180584 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.180549    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:04.180584 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.180564    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:04.180670 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.180626    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.180606603 +0000 UTC m=+42.612489063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:04.181109 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.181014    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:04.181109 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.181064    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.181048749 +0000 UTC m=+42.612931209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:04.281807 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.281769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:04.281974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.281825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:04.281974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.281857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:04.281974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.281929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:04.281974 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.281929    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:04.282223 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282009    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.281990863 +0000 UTC m=+42.713873325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:04.282223 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282011    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:04.282223 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282069    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.28205229 +0000 UTC m=+42.713934766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:04.282223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.282099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:04.282223 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282221    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:04.282492 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282264    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.282245902 +0000 UTC m=+42.714128375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:04.282492 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282318    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:04.282492 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282345    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.282336544 +0000 UTC m=+42.714219005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:04.282492 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:04.282401    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:08.282387848 +0000 UTC m=+42.714270311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:04.310722 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.310662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl" event={"ID":"241109d5-1076-4d7c-bedb-0a1b70487d17","Type":"ContainerStarted","Data":"e91d20ec953f3c38ca0507b3ba2570d4e613ba59875a70e61a52f0bce9c154c7"}
Jul 29 01:13:04.318246 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.318217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qr45k" event={"ID":"6741d76d-6d73-4dad-b5ab-2cb9f3525cfd","Type":"ContainerStarted","Data":"81edb94fb53ed8691ec4946a1840ef46ff76b1bfc180327bf88fff3a21192632"}
Jul 29 01:13:04.325689 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.325637    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-djlvl" podStartSLOduration=34.318994563 podStartE2EDuration="36.325618693s" totalImagesPullingTime="2.00662413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:04.324910705 +0000 UTC m=+38.756793183" watchObservedRunningTime="2026-07-29 01:13:04.325618693 +0000 UTC m=+38.757501174"
Jul 29 01:13:04.347181 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:04.347125    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qr45k" podStartSLOduration=9.943699104 podStartE2EDuration="38.347109558s" totalImagesPullingTime="28.403410454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:04.345064348 +0000 UTC m=+38.776946829" watchObservedRunningTime="2026-07-29 01:13:04.347109558 +0000 UTC m=+38.778992046"
Jul 29 01:13:08.217525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.217486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:08.217525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.217533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:08.218138 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.217683    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:08.218138 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.217734    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:08.218138 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.217755    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:08.218138 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.217759    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.21774348 +0000 UTC m=+50.649625938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:08.218138 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.217810    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.217792895 +0000 UTC m=+50.649675373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:08.318606 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.318572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:08.318794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.318627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:08.318794 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318704    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:08.318794 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318780    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.318758946 +0000 UTC m=+50.750641427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:08.318794 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318786    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318834    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.318818119 +0000 UTC m=+50.750700578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.318713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318847    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.318879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318902    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.318886356 +0000 UTC m=+50.750768830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318936    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:08.318946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.318964    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.318955702 +0000 UTC m=+50.750838160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:08.319033 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:08.319022    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:16.319010883 +0000 UTC m=+50.750893342 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:10.339505 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.339468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wkb2l" event={"ID":"49d40c02-c488-48a4-820d-257a0d4ee7b3","Type":"ContainerStarted","Data":"e5fa3a237ed506832c7639f6fe9f3ba2dd6e69de5b03ea8edb3d792ef04e4385"}
Jul 29 01:13:10.341551 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.341525    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-45szv" event={"ID":"04414c82-3152-4607-ae22-cae517c84cd4","Type":"ContainerStarted","Data":"c440896390efaacd7d64f9734ed334e570e03ff1f3624242c1ae2a533e15d94f"}
Jul 29 01:13:10.343244 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.343217    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/0.log"
Jul 29 01:13:10.343366 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.343255    2564 generic.go:356] "Generic (PLEG): container finished" podID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4" containerID="f7f540be1882093151836d01a3e3ba07277b9505856acae8896f7b5aa9994dbf" exitCode=255
Jul 29 01:13:10.343512 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.343462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" event={"ID":"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4","Type":"ContainerDied","Data":"f7f540be1882093151836d01a3e3ba07277b9505856acae8896f7b5aa9994dbf"}
Jul 29 01:13:10.343622 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.343549    2564 scope.go:117] "RemoveContainer" containerID="f7f540be1882093151836d01a3e3ba07277b9505856acae8896f7b5aa9994dbf"
Jul 29 01:13:10.345993 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.345969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v" event={"ID":"3dab08fa-1154-4caa-80f3-ecfd50ee4351","Type":"ContainerStarted","Data":"76147502da9749e66b3677c2fbbaf858ae7743af35aa34b45b6ee3d81f5f48f9"}
Jul 29 01:13:10.347847 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.347822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t" event={"ID":"53c70359-d905-4b35-a16f-6dd4d4443f68","Type":"ContainerStarted","Data":"54810e1855a115658a883a86724dd10699856994b9131459748d4462a0f0b7d3"}
Jul 29 01:13:10.349322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.349301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl" event={"ID":"711d634c-0515-4f70-9f82-6f387aeb020a","Type":"ContainerStarted","Data":"6c6e2d868ef57f50e303ecd2690cd405480d4129ad1333026c405ecad8cf0411"}
Jul 29 01:13:10.359943 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.359888    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-wkb2l" podStartSLOduration=34.598445064 podStartE2EDuration="42.359869854s" totalImagesPullingTime="7.76142479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:10.356847654 +0000 UTC m=+44.788730151" watchObservedRunningTime="2026-07-29 01:13:10.359869854 +0000 UTC m=+44.791752316"
Jul 29 01:13:10.378021 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.377957    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v" podStartSLOduration=34.618941417 podStartE2EDuration="42.377937244s" totalImagesPullingTime="7.758995827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:10.377307606 +0000 UTC m=+44.809190088" watchObservedRunningTime="2026-07-29 01:13:10.377937244 +0000 UTC m=+44.809819726"
Jul 29 01:13:10.430911 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.430860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-45szv" podStartSLOduration=2.664069901 podStartE2EDuration="10.430841658s" totalImagesPullingTime="7.766771757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:10.411457468 +0000 UTC m=+44.843339948" watchObservedRunningTime="2026-07-29 01:13:10.430841658 +0000 UTC m=+44.862724140"
Jul 29 01:13:10.431284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.431245    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl" podStartSLOduration=34.671909848 podStartE2EDuration="42.431236667s" totalImagesPullingTime="7.759326819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:10.429631523 +0000 UTC m=+44.861513998" watchObservedRunningTime="2026-07-29 01:13:10.431236667 +0000 UTC m=+44.863119146"
Jul 29 01:13:10.723954 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.723843    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-d659t" podStartSLOduration=34.966076908 podStartE2EDuration="42.723821344s" totalImagesPullingTime="7.757744436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:10.45197724 +0000 UTC m=+44.883859722" watchObservedRunningTime="2026-07-29 01:13:10.723821344 +0000 UTC m=+45.155703826"
Jul 29 01:13:10.725246 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.725214    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fbnhs"]
Jul 29 01:13:10.741456 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.741417    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fbnhs"]
Jul 29 01:13:10.741601 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.741542    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.744175 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.744088    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 01:13:10.744421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.744327    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 01:13:10.744538 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.744256    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z5fvs\""
Jul 29 01:13:10.798490 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.798447    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:10.798647 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.798518    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:10.844296 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.844264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.844296 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.844300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3714956-ef11-4e19-810a-0dbefd635a4d-crio-socket\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.844535 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.844323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.844535 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.844396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3714956-ef11-4e19-810a-0dbefd635a4d-data-volume\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.844636 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.844565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5ftp\" (UniqueName: \"kubernetes.io/projected/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-api-access-c5ftp\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945480 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ftp\" (UniqueName: \"kubernetes.io/projected/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-api-access-c5ftp\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945480 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3714956-ef11-4e19-810a-0dbefd635a4d-crio-socket\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:10.945641    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:10.945713    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:11.445693541 +0000 UTC m=+45.877576019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e3714956-ef11-4e19-810a-0dbefd635a4d-crio-socket\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:10.945815    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 01:13:10.945906 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:10.945866    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:11.44585313 +0000 UTC m=+45.877735593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : configmap "kube-rbac-proxy" not found
Jul 29 01:13:10.946323 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.945917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3714956-ef11-4e19-810a-0dbefd635a4d-data-volume\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.946375 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.946318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e3714956-ef11-4e19-810a-0dbefd635a4d-data-volume\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:10.955950 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:10.955927    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5ftp\" (UniqueName: \"kubernetes.io/projected/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-api-access-c5ftp\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:11.355863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.355838    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/1.log"
Jul 29 01:13:11.356316 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.356297    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/0.log"
Jul 29 01:13:11.356365 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.356328    2564 generic.go:356] "Generic (PLEG): container finished" podID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4" containerID="dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a" exitCode=255
Jul 29 01:13:11.356410 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.356366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" event={"ID":"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4","Type":"ContainerDied","Data":"dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a"}
Jul 29 01:13:11.356459 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.356432    2564 scope.go:117] "RemoveContainer" containerID="f7f540be1882093151836d01a3e3ba07277b9505856acae8896f7b5aa9994dbf"
Jul 29 01:13:11.356650 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.356628    2564 scope.go:117] "RemoveContainer" containerID="dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a"
Jul 29 01:13:11.356859 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:11.356834    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-kt2c4_openshift-console-operator(eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" podUID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4"
Jul 29 01:13:11.451554 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.451517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:11.451807 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:11.451779    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:11.451931 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:11.451839    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:12.451823447 +0000 UTC m=+46.883705908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:11.451931 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:11.451908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:11.452074 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:11.452001    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 01:13:11.452074 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:11.452060    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:12.4520448 +0000 UTC m=+46.883927271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : configmap "kube-rbac-proxy" not found
Jul 29 01:13:12.361677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:12.361650    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/1.log"
Jul 29 01:13:12.362030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:12.361936    2564 scope.go:117] "RemoveContainer" containerID="dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a"
Jul 29 01:13:12.362097 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:12.362078    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-kt2c4_openshift-console-operator(eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" podUID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4"
Jul 29 01:13:12.424090 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:12.424067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqr8t_1ea819b9-2b2d-4f81-aee1-d6e3636ee41a/dns-node-resolver/0.log"
Jul 29 01:13:12.460740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:12.460715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:12.460873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:12.460766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:12.460943 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:12.460892    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:12.460995 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:12.460968    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:14.460948961 +0000 UTC m=+48.892831426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:12.461070 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:12.460892    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 01:13:12.461127 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:12.461074    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:14.461061585 +0000 UTC m=+48.892944042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : configmap "kube-rbac-proxy" not found
Jul 29 01:13:13.222793 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:13.222765    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2l955_31ed3c06-7825-4d83-9da3-43877fb9780f/node-ca/0.log"
Jul 29 01:13:14.023473 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:14.023449    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-45szv_04414c82-3152-4607-ae22-cae517c84cd4/serve-healthcheck-canary/0.log"
Jul 29 01:13:14.477465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:14.477428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:14.477465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:14.477472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:14.477671 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:14.477543    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 01:13:14.477671 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:14.477594    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:18.477581716 +0000 UTC m=+52.909464174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : configmap "kube-rbac-proxy" not found
Jul 29 01:13:14.477746 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:14.477665    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:14.477746 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:14.477721    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:18.477707209 +0000 UTC m=+52.909589667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:16.294224 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.294190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:16.294224 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.294222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:16.294619 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.294327    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:16.294619 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.294337    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 01:13:16.294619 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.294350    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68f458fb64-hg287: secret "image-registry-tls" not found
Jul 29 01:13:16.294619 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.294389    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls podName:198bd2b4-3160-4256-a39f-db5f57868a3f nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.294374609 +0000 UTC m=+66.726257066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-qtsqf" (UID: "198bd2b4-3160-4256-a39f-db5f57868a3f") : secret "cluster-monitoring-operator-tls" not found
Jul 29 01:13:16.294619 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.294402    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls podName:66148de9-0a6a-4228-9ff0-cb215e865fe0 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.294396021 +0000 UTC m=+66.726278480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls") pod "image-registry-68f458fb64-hg287" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0") : secret "image-registry-tls" not found
Jul 29 01:13:16.394912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.394881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:16.395091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.394927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:16.395091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.394948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:16.395091 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395046    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.395028788 +0000 UTC m=+66.826911245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : configmap references non-existent config key: service-ca.crt
Jul 29 01:13:16.395091 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395045    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395113    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs podName:e147f4bd-b643-4c77-8ce9-5f7abfe38dec nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.395095945 +0000 UTC m=+66.826978404 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs") pod "router-default-857b7c775-7mj7f" (UID: "e147f4bd-b643-4c77-8ce9-5f7abfe38dec") : secret "router-metrics-certs-default" not found
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.395132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395136    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:16.395183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395212    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls podName:870dc954-5b03-4306-859f-484b01bb4701 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.395195907 +0000 UTC m=+66.827078369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls") pod "cluster-samples-operator-6748489656-nzrfl" (UID: "870dc954-5b03-4306-859f-484b01bb4701") : secret "samples-operator-tls" not found
Jul 29 01:13:16.395270 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395245    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 01:13:16.395466 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395286    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert podName:3d35aad0-55d6-4d74-a7d4-fb149201c916 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.3952746 +0000 UTC m=+66.827157060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fljjk" (UID: "3d35aad0-55d6-4d74-a7d4-fb149201c916") : secret "networking-console-plugin-cert" not found
Jul 29 01:13:16.395466 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395307    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 01:13:16.395466 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:16.395335    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls podName:95b3711a-bb50-4704-b92e-1bd02d314cd2 nodeName:}" failed. No retries permitted until 2026-07-29 01:13:32.395327816 +0000 UTC m=+66.827210277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls") pod "dns-default-lz9dv" (UID: "95b3711a-bb50-4704-b92e-1bd02d314cd2") : secret "dns-default-metrics-tls" not found
Jul 29 01:13:18.512649 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:18.512612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:18.512649 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:18.512655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:18.513173 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:18.512760    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:18.513173 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:18.512824    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls podName:e3714956-ef11-4e19-810a-0dbefd635a4d nodeName:}" failed. No retries permitted until 2026-07-29 01:13:26.512810205 +0000 UTC m=+60.944692662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fbnhs" (UID: "e3714956-ef11-4e19-810a-0dbefd635a4d") : secret "insights-runtime-extractor-tls" not found
Jul 29 01:13:18.513173 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:18.513128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e3714956-ef11-4e19-810a-0dbefd635a4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:20.797998 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:20.797964    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:20.798445 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:20.798045    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:20.798445 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:20.798391    2564 scope.go:117] "RemoveContainer" containerID="dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a"
Jul 29 01:13:21.391184 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391133    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:13:21.391548 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391534    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/1.log"
Jul 29 01:13:21.391610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391566    2564 generic.go:356] "Generic (PLEG): container finished" podID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4" containerID="1aa3057d52dd325d516e8c2e3a6385e107a98985bbb7796292ac00b8de892dbe" exitCode=255
Jul 29 01:13:21.391610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" event={"ID":"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4","Type":"ContainerDied","Data":"1aa3057d52dd325d516e8c2e3a6385e107a98985bbb7796292ac00b8de892dbe"}
Jul 29 01:13:21.391698 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391644    2564 scope.go:117] "RemoveContainer" containerID="dcffb1970a1e52b2908b460e75ae4c241d8002bdb884e673ef44afabf74d921a"
Jul 29 01:13:21.391931 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:21.391903    2564 scope.go:117] "RemoveContainer" containerID="1aa3057d52dd325d516e8c2e3a6385e107a98985bbb7796292ac00b8de892dbe"
Jul 29 01:13:21.392100 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:21.392079    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-kt2c4_openshift-console-operator(eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" podUID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4"
Jul 29 01:13:22.396250 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:22.396222    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:13:26.265594 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.265563    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nkp4z"
Jul 29 01:13:26.584652 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.584621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:26.586884 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.586858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e3714956-ef11-4e19-810a-0dbefd635a4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fbnhs\" (UID: \"e3714956-ef11-4e19-810a-0dbefd635a4d\") " pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:26.654996 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.654972    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z5fvs\""
Jul 29 01:13:26.663369 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.663352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fbnhs"
Jul 29 01:13:26.783314 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:26.783284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fbnhs"]
Jul 29 01:13:26.786064 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:26.786032    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3714956_ef11_4e19_810a_0dbefd635a4d.slice/crio-c78b1ab8bc7d87d63d8e7c76362b86a946e14ce3bc6a90dbfdd310a576e3724b WatchSource:0}: Error finding container c78b1ab8bc7d87d63d8e7c76362b86a946e14ce3bc6a90dbfdd310a576e3724b: Status 404 returned error can't find the container with id c78b1ab8bc7d87d63d8e7c76362b86a946e14ce3bc6a90dbfdd310a576e3724b
Jul 29 01:13:27.412670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:27.412622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fbnhs" event={"ID":"e3714956-ef11-4e19-810a-0dbefd635a4d","Type":"ContainerStarted","Data":"c78b1ab8bc7d87d63d8e7c76362b86a946e14ce3bc6a90dbfdd310a576e3724b"}
Jul 29 01:13:30.798175 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:30.798128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:30.798630 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:30.798195    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:30.798630 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:30.798474    2564 scope.go:117] "RemoveContainer" containerID="1aa3057d52dd325d516e8c2e3a6385e107a98985bbb7796292ac00b8de892dbe"
Jul 29 01:13:30.798756 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:13:30.798624    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-kt2c4_openshift-console-operator(eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" podUID="eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4"
Jul 29 01:13:31.732177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.732087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:31.732177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.732129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:13:31.734747 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.734725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h58zj\" (UniqueName: \"kubernetes.io/projected/7e9a5312-259b-45aa-baac-3ddd9425f781-kube-api-access-h58zj\") pod \"network-check-target-qw64x\" (UID: \"7e9a5312-259b-45aa-baac-3ddd9425f781\") " pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:31.734881 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.734851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 01:13:31.744236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.744209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e29ae934-2ad2-4b67-8d4c-27ca1d30a571-metrics-certs\") pod \"network-metrics-daemon-2tb8c\" (UID: \"e29ae934-2ad2-4b67-8d4c-27ca1d30a571\") " pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:13:31.832707 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.832684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:13:31.833008 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.832873    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nbzfd\""
Jul 29 01:13:31.834940 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.834923    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 01:13:31.840972 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.840957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tb8c"
Jul 29 01:13:31.844916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.844900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9438ab0c-aec4-41b1-b091-e541c119b72c-original-pull-secret\") pod \"global-pull-secret-syncer-sh9cm\" (UID: \"9438ab0c-aec4-41b1-b091-e541c119b72c\") " pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:13:31.848134 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.848112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82b2w\""
Jul 29 01:13:31.856072 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.856052    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:31.974721 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.974695    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tb8c"]
Jul 29 01:13:31.977608 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:31.977585    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29ae934_2ad2_4b67_8d4c_27ca1d30a571.slice/crio-bb28579fbe4c4e0f44f4633f1d5386b50bcfc3b429bea8ad240c069cdb36b09e WatchSource:0}: Error finding container bb28579fbe4c4e0f44f4633f1d5386b50bcfc3b429bea8ad240c069cdb36b09e: Status 404 returned error can't find the container with id bb28579fbe4c4e0f44f4633f1d5386b50bcfc3b429bea8ad240c069cdb36b09e
Jul 29 01:13:31.996055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:31.996008    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qw64x"]
Jul 29 01:13:31.998476 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:31.998453    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e9a5312_259b_45aa_baac_3ddd9425f781.slice/crio-1a3b3449464c3333f58d5093fcafe485dc1509147e0341b292a21dd71fa93182 WatchSource:0}: Error finding container 1a3b3449464c3333f58d5093fcafe485dc1509147e0341b292a21dd71fa93182: Status 404 returned error can't find the container with id 1a3b3449464c3333f58d5093fcafe485dc1509147e0341b292a21dd71fa93182
Jul 29 01:13:32.138685 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.138661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sh9cm"
Jul 29 01:13:32.261946 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.261871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sh9cm"]
Jul 29 01:13:32.264715 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:32.264691    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9438ab0c_aec4_41b1_b091_e541c119b72c.slice/crio-c84096686edac08cccf48e0006081da9e1aa17a2ad6f27db86d06760b67a7d94 WatchSource:0}: Error finding container c84096686edac08cccf48e0006081da9e1aa17a2ad6f27db86d06760b67a7d94: Status 404 returned error can't find the container with id c84096686edac08cccf48e0006081da9e1aa17a2ad6f27db86d06760b67a7d94
Jul 29 01:13:32.338009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.337977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:32.338009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.338011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:32.340384 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.340363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/198bd2b4-3160-4256-a39f-db5f57868a3f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-qtsqf\" (UID: \"198bd2b4-3160-4256-a39f-db5f57868a3f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:32.340487 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.340400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"image-registry-68f458fb64-hg287\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") " pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:32.431764 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.431730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sh9cm" event={"ID":"9438ab0c-aec4-41b1-b091-e541c119b72c","Type":"ContainerStarted","Data":"c84096686edac08cccf48e0006081da9e1aa17a2ad6f27db86d06760b67a7d94"}
Jul 29 01:13:32.433327 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.433300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qw64x" event={"ID":"7e9a5312-259b-45aa-baac-3ddd9425f781","Type":"ContainerStarted","Data":"eece3c636e988010e9cea7cfd5b64ebe08eb4fe9fc208ce936c2828f62ea4fd2"}
Jul 29 01:13:32.433441 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.433348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qw64x" event={"ID":"7e9a5312-259b-45aa-baac-3ddd9425f781","Type":"ContainerStarted","Data":"1a3b3449464c3333f58d5093fcafe485dc1509147e0341b292a21dd71fa93182"}
Jul 29 01:13:32.433501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.433482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:13:32.435025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.434998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fbnhs" event={"ID":"e3714956-ef11-4e19-810a-0dbefd635a4d","Type":"ContainerStarted","Data":"3278a938e5fec4d6d54a0acf4b6b6caae74eefc837c208558c4d32232d5a60c4"}
Jul 29 01:13:32.436203 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.436182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tb8c" event={"ID":"e29ae934-2ad2-4b67-8d4c-27ca1d30a571","Type":"ContainerStarted","Data":"bb28579fbe4c4e0f44f4633f1d5386b50bcfc3b429bea8ad240c069cdb36b09e"}
Jul 29 01:13:32.438692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.438666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:32.438779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.438700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:32.438779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.438721    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:32.438779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.438769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:32.439105 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.439082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:32.439636 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.439611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-service-ca-bundle\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:32.441509 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.441486    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/870dc954-5b03-4306-859f-484b01bb4701-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nzrfl\" (UID: \"870dc954-5b03-4306-859f-484b01bb4701\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:32.441704 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.441689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e147f4bd-b643-4c77-8ce9-5f7abfe38dec-metrics-certs\") pod \"router-default-857b7c775-7mj7f\" (UID: \"e147f4bd-b643-4c77-8ce9-5f7abfe38dec\") " pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:32.441750 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.441722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b3711a-bb50-4704-b92e-1bd02d314cd2-metrics-tls\") pod \"dns-default-lz9dv\" (UID: \"95b3711a-bb50-4704-b92e-1bd02d314cd2\") " pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:32.441828 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.441805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3d35aad0-55d6-4d74-a7d4-fb149201c916-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fljjk\" (UID: \"3d35aad0-55d6-4d74-a7d4-fb149201c916\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:32.449037 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.448995    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qw64x" podStartSLOduration=66.448981865 podStartE2EDuration="1m6.448981865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:32.448519945 +0000 UTC m=+66.880402425" watchObservedRunningTime="2026-07-29 01:13:32.448981865 +0000 UTC m=+66.880864323"
Jul 29 01:13:32.526618 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.526594    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sgklk\""
Jul 29 01:13:32.534802 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.534776    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:32.547913 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.547889    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vbv74\""
Jul 29 01:13:32.556055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.556031    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"
Jul 29 01:13:32.590910 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.590860    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n8mzz\""
Jul 29 01:13:32.598484 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.598458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:32.621258 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.621232    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xbshl\""
Jul 29 01:13:32.627959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.627733    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fxl8k\""
Jul 29 01:13:32.629591 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.629272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"
Jul 29 01:13:32.635373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.635332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"
Jul 29 01:13:32.640325 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.640307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rd2vs\""
Jul 29 01:13:32.650266 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.648120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:32.718373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.716457    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:13:32.738924 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.738838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf"]
Jul 29 01:13:32.862776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.862690    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl"]
Jul 29 01:13:32.890464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.890383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lz9dv"]
Jul 29 01:13:32.903078 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.903003    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fljjk"]
Jul 29 01:13:32.931458 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:32.931432    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-857b7c775-7mj7f"]
Jul 29 01:13:33.182486 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:33.179717    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d35aad0_55d6_4d74_a7d4_fb149201c916.slice/crio-94b57053d23a1fda01e274af3f4992be312927970805209b7b324466873f2065 WatchSource:0}: Error finding container 94b57053d23a1fda01e274af3f4992be312927970805209b7b324466873f2065: Status 404 returned error can't find the container with id 94b57053d23a1fda01e274af3f4992be312927970805209b7b324466873f2065
Jul 29 01:13:33.182486 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:33.181272    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode147f4bd_b643_4c77_8ce9_5f7abfe38dec.slice/crio-c59826223020ed0e145f20b9b9a15b1935325ad77be30f51b4160715dffdd1a1 WatchSource:0}: Error finding container c59826223020ed0e145f20b9b9a15b1935325ad77be30f51b4160715dffdd1a1: Status 404 returned error can't find the container with id c59826223020ed0e145f20b9b9a15b1935325ad77be30f51b4160715dffdd1a1
Jul 29 01:13:33.182860 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:33.182790    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b3711a_bb50_4704_b92e_1bd02d314cd2.slice/crio-39688784db4250c0e50618f9f91e56d60675d391660f07eaa0df20042ce78bfb WatchSource:0}: Error finding container 39688784db4250c0e50618f9f91e56d60675d391660f07eaa0df20042ce78bfb: Status 404 returned error can't find the container with id 39688784db4250c0e50618f9f91e56d60675d391660f07eaa0df20042ce78bfb
Jul 29 01:13:33.183728 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:33.183434    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66148de9_0a6a_4228_9ff0_cb215e865fe0.slice/crio-fef03715340dc076d242634d6cbef05b47bc3b7c11ef1b60a417d5e1e3a77bcd WatchSource:0}: Error finding container fef03715340dc076d242634d6cbef05b47bc3b7c11ef1b60a417d5e1e3a77bcd: Status 404 returned error can't find the container with id fef03715340dc076d242634d6cbef05b47bc3b7c11ef1b60a417d5e1e3a77bcd
Jul 29 01:13:33.184343 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:33.184169    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198bd2b4_3160_4256_a39f_db5f57868a3f.slice/crio-eeb123749316bc36ef562c1147d91c68ebc83dd2996c5513b632f18cee7c9bc6 WatchSource:0}: Error finding container eeb123749316bc36ef562c1147d91c68ebc83dd2996c5513b632f18cee7c9bc6: Status 404 returned error can't find the container with id eeb123749316bc36ef562c1147d91c68ebc83dd2996c5513b632f18cee7c9bc6
Jul 29 01:13:33.443750 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.443597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf" event={"ID":"198bd2b4-3160-4256-a39f-db5f57868a3f","Type":"ContainerStarted","Data":"eeb123749316bc36ef562c1147d91c68ebc83dd2996c5513b632f18cee7c9bc6"}
Jul 29 01:13:33.445916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.445790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f458fb64-hg287" event={"ID":"66148de9-0a6a-4228-9ff0-cb215e865fe0","Type":"ContainerStarted","Data":"f27bf19b2421cb55de2622b489017931526600b9446a9c970fcfec02c841db10"}
Jul 29 01:13:33.445916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.445838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f458fb64-hg287" event={"ID":"66148de9-0a6a-4228-9ff0-cb215e865fe0","Type":"ContainerStarted","Data":"fef03715340dc076d242634d6cbef05b47bc3b7c11ef1b60a417d5e1e3a77bcd"}
Jul 29 01:13:33.445916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.445884    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:33.447584 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.447516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl" event={"ID":"870dc954-5b03-4306-859f-484b01bb4701","Type":"ContainerStarted","Data":"3ad25506a47f0654e691c453ed8eefa0c785e68712296ee92733a19220a0d46b"}
Jul 29 01:13:33.450412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.450345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lz9dv" event={"ID":"95b3711a-bb50-4704-b92e-1bd02d314cd2","Type":"ContainerStarted","Data":"39688784db4250c0e50618f9f91e56d60675d391660f07eaa0df20042ce78bfb"}
Jul 29 01:13:33.453003 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.452981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-857b7c775-7mj7f" event={"ID":"e147f4bd-b643-4c77-8ce9-5f7abfe38dec","Type":"ContainerStarted","Data":"bf8750eeab16ab026d7d82cfa0615a25a8b72b1eed60fa4a519059d4d269d4ad"}
Jul 29 01:13:33.453114 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.453019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-857b7c775-7mj7f" event={"ID":"e147f4bd-b643-4c77-8ce9-5f7abfe38dec","Type":"ContainerStarted","Data":"c59826223020ed0e145f20b9b9a15b1935325ad77be30f51b4160715dffdd1a1"}
Jul 29 01:13:33.455548 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.455526    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk" event={"ID":"3d35aad0-55d6-4d74-a7d4-fb149201c916","Type":"ContainerStarted","Data":"94b57053d23a1fda01e274af3f4992be312927970805209b7b324466873f2065"}
Jul 29 01:13:33.469161 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.469102    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68f458fb64-hg287" podStartSLOduration=67.469083455 podStartE2EDuration="1m7.469083455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:33.46790442 +0000 UTC m=+67.899786899" watchObservedRunningTime="2026-07-29 01:13:33.469083455 +0000 UTC m=+67.900965936"
Jul 29 01:13:33.489401 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.489351    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-857b7c775-7mj7f" podStartSLOduration=65.489336924 podStartE2EDuration="1m5.489336924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:33.489175114 +0000 UTC m=+67.921057597" watchObservedRunningTime="2026-07-29 01:13:33.489336924 +0000 UTC m=+67.921219403"
Jul 29 01:13:33.649081 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.649020    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:33.652261 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:33.652211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:34.470020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:34.469193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tb8c" event={"ID":"e29ae934-2ad2-4b67-8d4c-27ca1d30a571","Type":"ContainerStarted","Data":"0e9c4eba18066c3b0a2095d9785e5e66c0682f53693f7817f1568f2a0d7a00de"}
Jul 29 01:13:34.470020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:34.469249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tb8c" event={"ID":"e29ae934-2ad2-4b67-8d4c-27ca1d30a571","Type":"ContainerStarted","Data":"d2c38b649ab425442d8852f8a51aa4ddf6905894a2d9f85c16a0c6200b45a2e1"}
Jul 29 01:13:34.470020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:34.469949    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:34.471637 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:34.471362    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-857b7c775-7mj7f"
Jul 29 01:13:34.496734 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:34.495464    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2tb8c" podStartSLOduration=66.926858435 podStartE2EDuration="1m8.495442634s" totalImagesPullingTime="1.568584199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:34.494097595 +0000 UTC m=+68.925980075" watchObservedRunningTime="2026-07-29 01:13:34.495442634 +0000 UTC m=+68.927325114"
Jul 29 01:13:41.515222 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.515180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf" event={"ID":"198bd2b4-3160-4256-a39f-db5f57868a3f","Type":"ContainerStarted","Data":"6c9cf5fced3909feaedfd24c40af236481ece7ebf76eb630ee8594c4d9293c2f"}
Jul 29 01:13:41.518094 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.518064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fbnhs" event={"ID":"e3714956-ef11-4e19-810a-0dbefd635a4d","Type":"ContainerStarted","Data":"74389d7517ae07b3790d444d76d83e01d01ed65a6159bb17ffef0010f6e8209d"}
Jul 29 01:13:41.521288 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.521201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl" event={"ID":"870dc954-5b03-4306-859f-484b01bb4701","Type":"ContainerStarted","Data":"b4dffe6eb3c6bcba8767c0bac38efb749895a8efa92ebf073be528b3d212e2da"}
Jul 29 01:13:41.522875 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.522797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lz9dv" event={"ID":"95b3711a-bb50-4704-b92e-1bd02d314cd2","Type":"ContainerStarted","Data":"f9dfef3c301e49b69836341f0aae51d41ef588dd62815162d35267e6603a82c0"}
Jul 29 01:13:41.526082 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.526053    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sh9cm" event={"ID":"9438ab0c-aec4-41b1-b091-e541c119b72c","Type":"ContainerStarted","Data":"5a8996d540e9d3b533c930b1683a9488705e6af27c5b220f8864c19d06a7f002"}
Jul 29 01:13:41.534260 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.534212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk" event={"ID":"3d35aad0-55d6-4d74-a7d4-fb149201c916","Type":"ContainerStarted","Data":"622d94b99f982ebba7d77fd23669455e436a1115c9b5507b1bf45c581f516366"}
Jul 29 01:13:41.535341 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.534883    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-qtsqf" podStartSLOduration=65.404887428 podStartE2EDuration="1m13.534866482s" totalImagesPullingTime="8.129979054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:41.533511758 +0000 UTC m=+75.965394239" watchObservedRunningTime="2026-07-29 01:13:41.534866482 +0000 UTC m=+75.966748964"
Jul 29 01:13:41.570033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.569972    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sh9cm" podStartSLOduration=66.490319724 podStartE2EDuration="1m15.569949977s" totalImagesPullingTime="9.079630253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:41.553047505 +0000 UTC m=+75.984929985" watchObservedRunningTime="2026-07-29 01:13:41.569949977 +0000 UTC m=+76.001832436"
Jul 29 01:13:41.570477 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:41.570443    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fljjk" podStartSLOduration=64.435746564 podStartE2EDuration="1m12.570427919s" totalImagesPullingTime="8.134681355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:41.569252163 +0000 UTC m=+76.001134646" watchObservedRunningTime="2026-07-29 01:13:41.570427919 +0000 UTC m=+76.002310400"
Jul 29 01:13:42.541836 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:42.541583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl" event={"ID":"870dc954-5b03-4306-859f-484b01bb4701","Type":"ContainerStarted","Data":"532e0e16f0387503163448ac909da3808db5c5d5f6a08a3fb34a30398c35253e"}
Jul 29 01:13:42.543674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:42.543644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lz9dv" event={"ID":"95b3711a-bb50-4704-b92e-1bd02d314cd2","Type":"ContainerStarted","Data":"b3aa21289a426a1bce457bd6cb78f19c21c927d2b47b92d0011495e55d9090cf"}
Jul 29 01:13:42.543798 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:42.543718    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:42.557975 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:42.557932    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nzrfl" podStartSLOduration=66.488033824 podStartE2EDuration="1m14.557916533s" totalImagesPullingTime="8.069882709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:42.556932863 +0000 UTC m=+76.988815342" watchObservedRunningTime="2026-07-29 01:13:42.557916533 +0000 UTC m=+76.989799014"
Jul 29 01:13:42.573392 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:42.573350    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lz9dv" podStartSLOduration=34.438299623 podStartE2EDuration="42.573335015s" totalImagesPullingTime="8.135035392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:42.572931549 +0000 UTC m=+77.004814032" watchObservedRunningTime="2026-07-29 01:13:42.573335015 +0000 UTC m=+77.005217495"
Jul 29 01:13:45.111115 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.111036    2564 scope.go:117] "RemoveContainer" containerID="1aa3057d52dd325d516e8c2e3a6385e107a98985bbb7796292ac00b8de892dbe"
Jul 29 01:13:45.559823 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.559795    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:13:45.559964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.559897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" event={"ID":"eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4","Type":"ContainerStarted","Data":"430a64bfaa3fa9d8ee8d93216f8e5c8bef052b0d8197a2361d529fc803b0f563"}
Jul 29 01:13:45.560386 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.560227    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:45.562351 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.562327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fbnhs" event={"ID":"e3714956-ef11-4e19-810a-0dbefd635a4d","Type":"ContainerStarted","Data":"1e1ae6950716fb7bd183a5fa6cbffc5a82beb81ed617fcc98af4179efe618a2a"}
Jul 29 01:13:45.581167 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.578618    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4" podStartSLOduration=69.827169719 podStartE2EDuration="1m17.578600606s" totalImagesPullingTime="7.751430887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:45.577898953 +0000 UTC m=+80.009781431" watchObservedRunningTime="2026-07-29 01:13:45.578600606 +0000 UTC m=+80.010483087"
Jul 29 01:13:45.730613 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.730587    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-kt2c4"
Jul 29 01:13:45.747357 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:45.747311    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fbnhs" podStartSLOduration=17.847751123 podStartE2EDuration="35.747294624s" totalImagesPullingTime="17.899543501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:13:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:45.597741264 +0000 UTC m=+80.029623752" watchObservedRunningTime="2026-07-29 01:13:45.747294624 +0000 UTC m=+80.179177104"
Jul 29 01:13:51.322648 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.322548    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"]
Jul 29 01:13:51.325746 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.325726    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-88zf9"]
Jul 29 01:13:51.325863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.325835    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.328178 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.328160    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.329048 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.329032    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 01:13:51.329138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.329081    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 01:13:51.330211 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.330191    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 01:13:51.330309 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.330224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 01:13:51.330309 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.330277    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zlzqm\""
Jul 29 01:13:51.330561 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.330546    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 01:13:51.330773 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.330756    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 01:13:51.331029 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.331015    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9pv8q\""
Jul 29 01:13:51.331369 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.331353    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 01:13:51.341301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.341281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"]
Jul 29 01:13:51.382660 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.382761 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gffn9\" (UniqueName: \"kubernetes.io/projected/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-kube-api-access-gffn9\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.382761 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-root\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.382761 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-tls\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.382761 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pt6r\" (UniqueName: \"kubernetes.io/projected/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-api-access-7pt6r\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.382898 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-metrics-client-ca\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.382898 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.382898 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.383025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.383025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.383025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-wtmp\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.383025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.383025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.382998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.383355 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.383042    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-sys\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.383355 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.383073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-textfile\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.483938 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.483911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-metrics-client-ca\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484034 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.483946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484034 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.483967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484034 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484183 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484183 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-wtmp\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484183 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484183 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-sys\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-textfile\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-wtmp\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gffn9\" (UniqueName: \"kubernetes.io/projected/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-kube-api-access-gffn9\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-root\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-tls\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pt6r\" (UniqueName: \"kubernetes.io/projected/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-api-access-7pt6r\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-metrics-client-ca\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-root\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484697    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.484725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-sys\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.485018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.484731    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-textfile\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.485104 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.485079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.485186 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.485107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.486898 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.486878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.487014 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.486993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-node-exporter-tls\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.487208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.487188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.487208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.487199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.492535 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.492510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gffn9\" (UniqueName: \"kubernetes.io/projected/80853a60-2d6a-4dc7-b7bb-10ea0d05af6c-kube-api-access-gffn9\") pod \"node-exporter-88zf9\" (UID: \"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c\") " pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.492804 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.492787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pt6r\" (UniqueName: \"kubernetes.io/projected/8466b7f3-cff6-43c8-98c7-7fb92ddbf875-kube-api-access-7pt6r\") pod \"kube-state-metrics-56d9f944b4-xdphk\" (UID: \"8466b7f3-cff6-43c8-98c7-7fb92ddbf875\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.636346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.636268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"
Jul 29 01:13:51.642039 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.642021    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88zf9"
Jul 29 01:13:51.651776 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:51.651750    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80853a60_2d6a_4dc7_b7bb_10ea0d05af6c.slice/crio-6d3a0223cca7192f3b9d7bb17fadd27f04830e403a9993ed09ddf7bab37f76e1 WatchSource:0}: Error finding container 6d3a0223cca7192f3b9d7bb17fadd27f04830e403a9993ed09ddf7bab37f76e1: Status 404 returned error can't find the container with id 6d3a0223cca7192f3b9d7bb17fadd27f04830e403a9993ed09ddf7bab37f76e1
Jul 29 01:13:51.759020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:51.758931    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk"]
Jul 29 01:13:51.761450 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:13:51.761425    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8466b7f3_cff6_43c8_98c7_7fb92ddbf875.slice/crio-ac70412c394e42320494888f475e15f22f989d6d915b2843ff62f6df01e6d10a WatchSource:0}: Error finding container ac70412c394e42320494888f475e15f22f989d6d915b2843ff62f6df01e6d10a: Status 404 returned error can't find the container with id ac70412c394e42320494888f475e15f22f989d6d915b2843ff62f6df01e6d10a
Jul 29 01:13:52.551704 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:52.551674    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lz9dv"
Jul 29 01:13:52.595016 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:52.594982    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk" event={"ID":"8466b7f3-cff6-43c8-98c7-7fb92ddbf875","Type":"ContainerStarted","Data":"ac70412c394e42320494888f475e15f22f989d6d915b2843ff62f6df01e6d10a"}
Jul 29 01:13:52.596963 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:52.596905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zf9" event={"ID":"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c","Type":"ContainerStarted","Data":"6d3a0223cca7192f3b9d7bb17fadd27f04830e403a9993ed09ddf7bab37f76e1"}
Jul 29 01:13:53.607521 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.607485    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk" event={"ID":"8466b7f3-cff6-43c8-98c7-7fb92ddbf875","Type":"ContainerStarted","Data":"00652513f24765a472cee78cb1efce8fb9b0c44a2166114447f3837e7fe98562"}
Jul 29 01:13:53.607957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.607584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk" event={"ID":"8466b7f3-cff6-43c8-98c7-7fb92ddbf875","Type":"ContainerStarted","Data":"1dedc2a96ec275d6329736155dc7fafdca86b87e4e4e2380b9cce17a146df26d"}
Jul 29 01:13:53.607957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.607610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk" event={"ID":"8466b7f3-cff6-43c8-98c7-7fb92ddbf875","Type":"ContainerStarted","Data":"31cbacc3e4110c914d7b5083b8ce13c8183fddd56d0d14a1631b7d25afd12ef1"}
Jul 29 01:13:53.609187 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.609135    2564 generic.go:356] "Generic (PLEG): container finished" podID="80853a60-2d6a-4dc7-b7bb-10ea0d05af6c" containerID="265a99fd82f8d3217fde3913e154ad9666761fef00decf9cbc27b772cd9761eb" exitCode=0
Jul 29 01:13:53.609187 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.609177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zf9" event={"ID":"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c","Type":"ContainerDied","Data":"265a99fd82f8d3217fde3913e154ad9666761fef00decf9cbc27b772cd9761eb"}
Jul 29 01:13:53.646417 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:53.646376    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-xdphk" podStartSLOduration=1.151261483 podStartE2EDuration="2.646361485s" totalImagesPullingTime="1.495100002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:53.625607896 +0000 UTC m=+88.057490378" watchObservedRunningTime="2026-07-29 01:13:53.646361485 +0000 UTC m=+88.078243964"
Jul 29 01:13:54.473122 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:54.473093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:13:54.617194 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:54.617133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zf9" event={"ID":"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c","Type":"ContainerStarted","Data":"0a16bc4427385833bdd346bb6c1a7993265661d2cba1ff29f0f4dc108fd909d4"}
Jul 29 01:13:54.617589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:54.617223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zf9" event={"ID":"80853a60-2d6a-4dc7-b7bb-10ea0d05af6c","Type":"ContainerStarted","Data":"8e5afff5affbf85e8d867d45376d6a41b9fc59415d6f961e76b2534d7e7b7637"}
Jul 29 01:13:54.634091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:54.634048    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-88zf9" podStartSLOduration=2.393691244 podStartE2EDuration="3.634033178s" totalImagesPullingTime="1.170643241s" totalInitContainerRuntime="69.698693ms" isStatefulPod=true podCreationTimestamp="2026-07-29 01:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:13:54.633486108 +0000 UTC m=+89.065368589" watchObservedRunningTime="2026-07-29 01:13:54.634033178 +0000 UTC m=+89.065915660"
Jul 29 01:13:56.535359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:13:56.535325    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:14:03.457557 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:03.457524    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qw64x"
Jul 29 01:14:04.002068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.002039    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:14:04.145158 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.145130    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:14:04.145291 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.145249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.148165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.148089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jm8j4\""
Jul 29 01:14:04.148165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.148090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 01:14:04.148405 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.148388    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 01:14:04.149499 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.149453    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 01:14:04.149499 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.149480    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 01:14:04.149678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.149508    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 01:14:04.149678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.149543    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 01:14:04.149678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.149624    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 01:14:04.154220 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.154192    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 01:14:04.180449 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180538 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8dfx\" (UniqueName: \"kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180538 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180538 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.180679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.180673    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.281747 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.281873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8dfx\" (UniqueName: \"kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.281873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.281873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.281873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.281932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282566 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.282533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.282579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.282601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.282810 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.282719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.284130 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.284106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.284340 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.284320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.289502 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.289484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8dfx\" (UniqueName: \"kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx\") pod \"console-5d8b57b7c8-qmh2f\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") " pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.456082 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.456027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:04.603608 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.603580    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:14:04.607016 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:14:04.606988    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdf9958_97a4_445f_82f1_45a74fac02c8.slice/crio-eb18a2f72365f00ef7d8c0db677141752760b1babd4dd0bf24d13e08f54be370 WatchSource:0}: Error finding container eb18a2f72365f00ef7d8c0db677141752760b1babd4dd0bf24d13e08f54be370: Status 404 returned error can't find the container with id eb18a2f72365f00ef7d8c0db677141752760b1babd4dd0bf24d13e08f54be370
Jul 29 01:14:04.658110 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:04.658084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8b57b7c8-qmh2f" event={"ID":"6cdf9958-97a4-445f-82f1-45a74fac02c8","Type":"ContainerStarted","Data":"eb18a2f72365f00ef7d8c0db677141752760b1babd4dd0bf24d13e08f54be370"}
Jul 29 01:14:07.671382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:07.671340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8b57b7c8-qmh2f" event={"ID":"6cdf9958-97a4-445f-82f1-45a74fac02c8","Type":"ContainerStarted","Data":"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"}
Jul 29 01:14:07.690766 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:07.690714    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8b57b7c8-qmh2f" podStartSLOduration=2.067059869 podStartE2EDuration="4.690701401s" totalImagesPullingTime="2.623641532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:14:07.688734099 +0000 UTC m=+102.120616584" watchObservedRunningTime="2026-07-29 01:14:07.690701401 +0000 UTC m=+102.122583881"
Jul 29 01:14:14.457204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:14.457168    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:14.457548 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:14.457237    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:14.462107 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:14.462083    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:14.697697 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:14.697672    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:14:21.558605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.558559    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-68f458fb64-hg287" podUID="66148de9-0a6a-4228-9ff0-cb215e865fe0" containerName="registry" containerID="cri-o://f27bf19b2421cb55de2622b489017931526600b9446a9c970fcfec02c841db10" gracePeriod=30
Jul 29 01:14:21.722079 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.722040    2564 generic.go:356] "Generic (PLEG): container finished" podID="66148de9-0a6a-4228-9ff0-cb215e865fe0" containerID="f27bf19b2421cb55de2622b489017931526600b9446a9c970fcfec02c841db10" exitCode=0
Jul 29 01:14:21.722253 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.722110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f458fb64-hg287" event={"ID":"66148de9-0a6a-4228-9ff0-cb215e865fe0","Type":"ContainerDied","Data":"f27bf19b2421cb55de2622b489017931526600b9446a9c970fcfec02c841db10"}
Jul 29 01:14:21.803644 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.803622    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:14:21.911906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.911843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.911906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.911882    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912059 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.911951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8q4f\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912059 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.911998    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912059 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912029    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912059 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912278 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912076    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912278 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912098    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token\") pod \"66148de9-0a6a-4228-9ff0-cb215e865fe0\" (UID: \"66148de9-0a6a-4228-9ff0-cb215e865fe0\") "
Jul 29 01:14:21.912377 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912297    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:14:21.912937 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.912899    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:14:21.914517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.914482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:14:21.914635 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.914614    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f" (OuterVolumeSpecName: "kube-api-access-s8q4f") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "kube-api-access-s8q4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:14:21.914693 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.914669    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:14:21.914773 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.914750    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:14:21.914866 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.914789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:14:21.920433 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:21.920409    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "66148de9-0a6a-4228-9ff0-cb215e865fe0" (UID: "66148de9-0a6a-4228-9ff0-cb215e865fe0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:14:22.013340 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013316    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-trusted-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013340 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013339    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-bound-sa-token\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013348    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-certificates\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013359    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-image-registry-private-configuration\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013369    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8q4f\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-kube-api-access-s8q4f\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013377    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66148de9-0a6a-4228-9ff0-cb215e865fe0-registry-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013385    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66148de9-0a6a-4228-9ff0-cb215e865fe0-ca-trust-extracted\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.013495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.013393    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66148de9-0a6a-4228-9ff0-cb215e865fe0-installation-pull-secrets\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:14:22.726488 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.726457    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f458fb64-hg287"
Jul 29 01:14:22.726488 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.726475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f458fb64-hg287" event={"ID":"66148de9-0a6a-4228-9ff0-cb215e865fe0","Type":"ContainerDied","Data":"fef03715340dc076d242634d6cbef05b47bc3b7c11ef1b60a417d5e1e3a77bcd"}
Jul 29 01:14:22.726990 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.726526    2564 scope.go:117] "RemoveContainer" containerID="f27bf19b2421cb55de2622b489017931526600b9446a9c970fcfec02c841db10"
Jul 29 01:14:22.744495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.744472    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:14:22.749010 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:22.748991    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68f458fb64-hg287"]
Jul 29 01:14:24.115138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:24.115104    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66148de9-0a6a-4228-9ff0-cb215e865fe0" path="/var/lib/kubelet/pods/66148de9-0a6a-4228-9ff0-cb215e865fe0/volumes"
Jul 29 01:14:36.778532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:36.778497    2564 generic.go:356] "Generic (PLEG): container finished" podID="3dab08fa-1154-4caa-80f3-ecfd50ee4351" containerID="76147502da9749e66b3677c2fbbaf858ae7743af35aa34b45b6ee3d81f5f48f9" exitCode=0
Jul 29 01:14:36.778889 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:36.778571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v" event={"ID":"3dab08fa-1154-4caa-80f3-ecfd50ee4351","Type":"ContainerDied","Data":"76147502da9749e66b3677c2fbbaf858ae7743af35aa34b45b6ee3d81f5f48f9"}
Jul 29 01:14:36.778889 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:36.778856    2564 scope.go:117] "RemoveContainer" containerID="76147502da9749e66b3677c2fbbaf858ae7743af35aa34b45b6ee3d81f5f48f9"
Jul 29 01:14:37.783966 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:37.783934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gjw2v" event={"ID":"3dab08fa-1154-4caa-80f3-ecfd50ee4351","Type":"ContainerStarted","Data":"3140578089cb4ba01deabcbb2cb3e90e27f63e406bf9dac4c04e18e25b5cbdb8"}
Jul 29 01:14:40.795668 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:40.795635    2564 generic.go:356] "Generic (PLEG): container finished" podID="49d40c02-c488-48a4-820d-257a0d4ee7b3" containerID="e5fa3a237ed506832c7639f6fe9f3ba2dd6e69de5b03ea8edb3d792ef04e4385" exitCode=0
Jul 29 01:14:40.796069 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:40.795717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wkb2l" event={"ID":"49d40c02-c488-48a4-820d-257a0d4ee7b3","Type":"ContainerDied","Data":"e5fa3a237ed506832c7639f6fe9f3ba2dd6e69de5b03ea8edb3d792ef04e4385"}
Jul 29 01:14:40.796069 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:40.796062    2564 scope.go:117] "RemoveContainer" containerID="e5fa3a237ed506832c7639f6fe9f3ba2dd6e69de5b03ea8edb3d792ef04e4385"
Jul 29 01:14:41.800562 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:41.800528    2564 generic.go:356] "Generic (PLEG): container finished" podID="711d634c-0515-4f70-9f82-6f387aeb020a" containerID="6c6e2d868ef57f50e303ecd2690cd405480d4129ad1333026c405ecad8cf0411" exitCode=0
Jul 29 01:14:41.800967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:41.800601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl" event={"ID":"711d634c-0515-4f70-9f82-6f387aeb020a","Type":"ContainerDied","Data":"6c6e2d868ef57f50e303ecd2690cd405480d4129ad1333026c405ecad8cf0411"}
Jul 29 01:14:41.800967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:41.800919    2564 scope.go:117] "RemoveContainer" containerID="6c6e2d868ef57f50e303ecd2690cd405480d4129ad1333026c405ecad8cf0411"
Jul 29 01:14:41.802483 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:41.802461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wkb2l" event={"ID":"49d40c02-c488-48a4-820d-257a0d4ee7b3","Type":"ContainerStarted","Data":"0dfc5a1ab4fb830148858f74bb41cd12abd4c53e33e3f57c3d6d3085b6b81438"}
Jul 29 01:14:42.808634 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:14:42.808590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl" event={"ID":"711d634c-0515-4f70-9f82-6f387aeb020a","Type":"ContainerStarted","Data":"7bc3fe5120267e26bf940d01972dc04be8c16c586e8814f09044da9411fb8ba8"}
Jul 29 01:15:14.364597 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.364564    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:15:14.365015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.364853    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66148de9-0a6a-4228-9ff0-cb215e865fe0" containerName="registry"
Jul 29 01:15:14.365015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.364866    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66148de9-0a6a-4228-9ff0-cb215e865fe0" containerName="registry"
Jul 29 01:15:14.365015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.364921    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66148de9-0a6a-4228-9ff0-cb215e865fe0" containerName="registry"
Jul 29 01:15:14.368329 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.368308    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.380448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.380382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:15:14.480070 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9hgv\" (UniqueName: \"kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.480330 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.480281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581446 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9hgv\" (UniqueName: \"kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581586 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581796 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.581796 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.581674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.582473 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.582451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.582569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.582477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.582569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.582477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.582685 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.582663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.583975 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.583952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.584080 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.584062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.591993 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.591973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9hgv\" (UniqueName: \"kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv\") pod \"console-6b69cdc7fc-4jlps\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") " pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.679165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.679094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:14.803939 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.803911    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:15:14.806972 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:15:14.806948    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842952c0_4de9_4ac3_9d58_bca1a4cfca51.slice/crio-e7fe444394e9cffe4c588d0323a3d564802e2038dbec4b7db2256930ac754b2d WatchSource:0}: Error finding container e7fe444394e9cffe4c588d0323a3d564802e2038dbec4b7db2256930ac754b2d: Status 404 returned error can't find the container with id e7fe444394e9cffe4c588d0323a3d564802e2038dbec4b7db2256930ac754b2d
Jul 29 01:15:14.931178 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.931066    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69cdc7fc-4jlps" event={"ID":"842952c0-4de9-4ac3-9d58-bca1a4cfca51","Type":"ContainerStarted","Data":"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"}
Jul 29 01:15:14.931178 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.931117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69cdc7fc-4jlps" event={"ID":"842952c0-4de9-4ac3-9d58-bca1a4cfca51","Type":"ContainerStarted","Data":"e7fe444394e9cffe4c588d0323a3d564802e2038dbec4b7db2256930ac754b2d"}
Jul 29 01:15:14.950835 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:14.950789    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b69cdc7fc-4jlps" podStartSLOduration=0.950774446 podStartE2EDuration="950.774446ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:15:14.948930408 +0000 UTC m=+169.380812889" watchObservedRunningTime="2026-07-29 01:15:14.950774446 +0000 UTC m=+169.382656925"
Jul 29 01:15:24.679932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:24.679846    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:24.679932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:24.679908    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:24.684501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:24.684476    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:24.973176 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:24.973086    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:15:25.018928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:25.018895    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:15:50.039603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.039535    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d8b57b7c8-qmh2f" podUID="6cdf9958-97a4-445f-82f1-45a74fac02c8" containerName="console" containerID="cri-o://30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440" gracePeriod=15
Jul 29 01:15:50.275861 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.275840    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:15:50.338339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338344    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8dfx\" (UniqueName: \"kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338479    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338520    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert\") pod \"6cdf9958-97a4-445f-82f1-45a74fac02c8\" (UID: \"6cdf9958-97a4-445f-82f1-45a74fac02c8\") "
Jul 29 01:15:50.338794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338673    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config" (OuterVolumeSpecName: "console-config") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:15:50.338794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338781    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca" (OuterVolumeSpecName: "service-ca") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:15:50.338894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338829    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-config\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.338894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.338863    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:15:50.339066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.339045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:15:50.340540 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.340513    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:15:50.340639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.340562    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:15:50.340639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.340578    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx" (OuterVolumeSpecName: "kube-api-access-r8dfx") pod "6cdf9958-97a4-445f-82f1-45a74fac02c8" (UID: "6cdf9958-97a4-445f-82f1-45a74fac02c8"). InnerVolumeSpecName "kube-api-access-r8dfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:15:50.439607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439582    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.439607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439603    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-oauth-config\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.439736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439614    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8dfx\" (UniqueName: \"kubernetes.io/projected/6cdf9958-97a4-445f-82f1-45a74fac02c8-kube-api-access-r8dfx\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.439736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439624    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-trusted-ca-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.439736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439632    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cdf9958-97a4-445f-82f1-45a74fac02c8-console-serving-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:50.439736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:50.439640    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cdf9958-97a4-445f-82f1-45a74fac02c8-oauth-serving-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:15:51.068722 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.068687    2564 generic.go:356] "Generic (PLEG): container finished" podID="6cdf9958-97a4-445f-82f1-45a74fac02c8" containerID="30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440" exitCode=0
Jul 29 01:15:51.068722 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.068720    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8b57b7c8-qmh2f"
Jul 29 01:15:51.069131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.068765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8b57b7c8-qmh2f" event={"ID":"6cdf9958-97a4-445f-82f1-45a74fac02c8","Type":"ContainerDied","Data":"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"}
Jul 29 01:15:51.069131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.068816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8b57b7c8-qmh2f" event={"ID":"6cdf9958-97a4-445f-82f1-45a74fac02c8","Type":"ContainerDied","Data":"eb18a2f72365f00ef7d8c0db677141752760b1babd4dd0bf24d13e08f54be370"}
Jul 29 01:15:51.069131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.068842    2564 scope.go:117] "RemoveContainer" containerID="30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"
Jul 29 01:15:51.077397 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.077377    2564 scope.go:117] "RemoveContainer" containerID="30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"
Jul 29 01:15:51.077693 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:15:51.077674    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440\": container with ID starting with 30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440 not found: ID does not exist" containerID="30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"
Jul 29 01:15:51.077741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.077702    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440"} err="failed to get container status \"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440\": rpc error: code = NotFound desc = could not find container \"30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440\": container with ID starting with 30d033179385f99aa6a18e5ae9771a12c4f7be185488a98eb2a495adeee21440 not found: ID does not exist"
Jul 29 01:15:51.089927 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.089899    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:15:51.096215 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:51.096194    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8b57b7c8-qmh2f"]
Jul 29 01:15:52.115620 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:15:52.115586    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cdf9958-97a4-445f-82f1-45a74fac02c8" path="/var/lib/kubelet/pods/6cdf9958-97a4-445f-82f1-45a74fac02c8/volumes"
Jul 29 01:16:24.295434 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.295401    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"]
Jul 29 01:16:24.295860 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.295681    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdf9958-97a4-445f-82f1-45a74fac02c8" containerName="console"
Jul 29 01:16:24.295860 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.295693    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf9958-97a4-445f-82f1-45a74fac02c8" containerName="console"
Jul 29 01:16:24.295860 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.295749    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdf9958-97a4-445f-82f1-45a74fac02c8" containerName="console"
Jul 29 01:16:24.297757 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.297741    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.301572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.301547    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 01:16:24.301676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.301604    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 01:16:24.301755 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.301734    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tv544\""
Jul 29 01:16:24.309859 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.309838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"]
Jul 29 01:16:24.372196 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.372173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.372304 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.372201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.372304 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.372224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkpcg\" (UniqueName: \"kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.473014 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.472991    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.473116 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.473020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.473116 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.473041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkpcg\" (UniqueName: \"kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.473381 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.473363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.473419 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.473392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.481508 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.481484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkpcg\" (UniqueName: \"kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.606405 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.606347    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:24.731420 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:24.731284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"]
Jul 29 01:16:24.733891 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:16:24.733862    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d23b3f_e34c_49a4_8308_bd71e1f64541.slice/crio-76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2 WatchSource:0}: Error finding container 76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2: Status 404 returned error can't find the container with id 76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2
Jul 29 01:16:25.193412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:25.193378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc" event={"ID":"16d23b3f-e34c-49a4-8308-bd71e1f64541","Type":"ContainerStarted","Data":"76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2"}
Jul 29 01:16:31.218517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:31.218483    2564 generic.go:356] "Generic (PLEG): container finished" podID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerID="e2aca3da00f89c97b5172d9f347a52add17b053cc91232d45ca81d39b333128a" exitCode=0
Jul 29 01:16:31.218888 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:31.218572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc" event={"ID":"16d23b3f-e34c-49a4-8308-bd71e1f64541","Type":"ContainerDied","Data":"e2aca3da00f89c97b5172d9f347a52add17b053cc91232d45ca81d39b333128a"}
Jul 29 01:16:34.231943 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:34.231910    2564 generic.go:356] "Generic (PLEG): container finished" podID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerID="02bbe15193bae073d5046480e1a61f5182a26f890af12b1410601269c56ccd50" exitCode=0
Jul 29 01:16:34.232313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:34.231994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc" event={"ID":"16d23b3f-e34c-49a4-8308-bd71e1f64541","Type":"ContainerDied","Data":"02bbe15193bae073d5046480e1a61f5182a26f890af12b1410601269c56ccd50"}
Jul 29 01:16:40.259411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:40.259374    2564 generic.go:356] "Generic (PLEG): container finished" podID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerID="af16c05fd98531ea62755787c7936ce82194d32ebcbba6f4d8ab4e7b6b103fe7" exitCode=0
Jul 29 01:16:40.259699 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:40.259417    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc" event={"ID":"16d23b3f-e34c-49a4-8308-bd71e1f64541","Type":"ContainerDied","Data":"af16c05fd98531ea62755787c7936ce82194d32ebcbba6f4d8ab4e7b6b103fe7"}
Jul 29 01:16:41.388056 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.388033    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:41.519644 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.519566    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle\") pod \"16d23b3f-e34c-49a4-8308-bd71e1f64541\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") "
Jul 29 01:16:41.519644 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.519649    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkpcg\" (UniqueName: \"kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg\") pod \"16d23b3f-e34c-49a4-8308-bd71e1f64541\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") "
Jul 29 01:16:41.519828 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.519670    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util\") pod \"16d23b3f-e34c-49a4-8308-bd71e1f64541\" (UID: \"16d23b3f-e34c-49a4-8308-bd71e1f64541\") "
Jul 29 01:16:41.520354 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.520330    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle" (OuterVolumeSpecName: "bundle") pod "16d23b3f-e34c-49a4-8308-bd71e1f64541" (UID: "16d23b3f-e34c-49a4-8308-bd71e1f64541"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:16:41.521794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.521774    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg" (OuterVolumeSpecName: "kube-api-access-qkpcg") pod "16d23b3f-e34c-49a4-8308-bd71e1f64541" (UID: "16d23b3f-e34c-49a4-8308-bd71e1f64541"). InnerVolumeSpecName "kube-api-access-qkpcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:16:41.523957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.523933    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util" (OuterVolumeSpecName: "util") pod "16d23b3f-e34c-49a4-8308-bd71e1f64541" (UID: "16d23b3f-e34c-49a4-8308-bd71e1f64541"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:16:41.620887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.620861    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:41.620887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.620887    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16d23b3f-e34c-49a4-8308-bd71e1f64541-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:41.621023 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:41.620899    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkpcg\" (UniqueName: \"kubernetes.io/projected/16d23b3f-e34c-49a4-8308-bd71e1f64541-kube-api-access-qkpcg\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:42.269575 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:42.269545    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc"
Jul 29 01:16:42.269702 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:42.269546    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540bkksc" event={"ID":"16d23b3f-e34c-49a4-8308-bd71e1f64541","Type":"ContainerDied","Data":"76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2"}
Jul 29 01:16:42.269702 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:42.269674    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76055c07998fff07cfa436167c5e2a0b42d3f59c790f3cb5a87ca1f9796960c2"
Jul 29 01:16:47.300441 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.300408    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"]
Jul 29 01:16:47.301023 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301001    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="extract"
Jul 29 01:16:47.301135 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301112    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="extract"
Jul 29 01:16:47.301222 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301172    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="util"
Jul 29 01:16:47.301222 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301184    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="util"
Jul 29 01:16:47.301332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301223    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="pull"
Jul 29 01:16:47.301332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301247    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="pull"
Jul 29 01:16:47.301453 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.301437    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d23b3f-e34c-49a4-8308-bd71e1f64541" containerName="extract"
Jul 29 01:16:47.318167 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.318124    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.320734 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.320706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"]
Jul 29 01:16:47.321212 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.321196    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 01:16:47.322332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.322317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:16:47.322491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.322470    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-67sns\""
Jul 29 01:16:47.465248 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.465208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.465421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.465290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn2gb\" (UniqueName: \"kubernetes.io/projected/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-kube-api-access-rn2gb\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.566378 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.566306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.566378 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.566354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn2gb\" (UniqueName: \"kubernetes.io/projected/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-kube-api-access-rn2gb\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.566675 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.566656    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.576188 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.576165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn2gb\" (UniqueName: \"kubernetes.io/projected/4f46c5d4-215b-42a5-b6d0-2ece30018c3a-kube-api-access-rn2gb\") pod \"cert-manager-operator-controller-manager-55c444bf86-jt5fg\" (UID: \"4f46c5d4-215b-42a5-b6d0-2ece30018c3a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.628128 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.628096    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"
Jul 29 01:16:47.779659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:47.779627    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg"]
Jul 29 01:16:47.783281 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:16:47.783255    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f46c5d4_215b_42a5_b6d0_2ece30018c3a.slice/crio-a724cfe969faf6a9d37cfeb88ac679ef4d24fa197e63f57db848c61f76126f50 WatchSource:0}: Error finding container a724cfe969faf6a9d37cfeb88ac679ef4d24fa197e63f57db848c61f76126f50: Status 404 returned error can't find the container with id a724cfe969faf6a9d37cfeb88ac679ef4d24fa197e63f57db848c61f76126f50
Jul 29 01:16:48.293449 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:48.293417    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg" event={"ID":"4f46c5d4-215b-42a5-b6d0-2ece30018c3a","Type":"ContainerStarted","Data":"a724cfe969faf6a9d37cfeb88ac679ef4d24fa197e63f57db848c61f76126f50"}
Jul 29 01:16:50.303750 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:50.303717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg" event={"ID":"4f46c5d4-215b-42a5-b6d0-2ece30018c3a","Type":"ContainerStarted","Data":"fe574e497e9c13affde4746e8d9f48db12ced11649b22c1e169b7a41f31cb196"}
Jul 29 01:16:50.333177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:50.333118    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jt5fg" podStartSLOduration=0.966798505 podStartE2EDuration="3.333101862s" totalImagesPullingTime="2.366303357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:16:50.330942759 +0000 UTC m=+264.762825238" watchObservedRunningTime="2026-07-29 01:16:50.333101862 +0000 UTC m=+264.764984341"
Jul 29 01:16:51.291031 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.291000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nr2wr"]
Jul 29 01:16:51.315900 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.315873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.316727 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.316704    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nr2wr"]
Jul 29 01:16:51.318683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.318662    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 01:16:51.319220 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.319203    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 01:16:51.415893 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.415869    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"]
Jul 29 01:16:51.436795 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.436772    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"]
Jul 29 01:16:51.436903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.436856    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.439617 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.439586    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 01:16:51.439617 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.439613    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 01:16:51.439751 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.439613    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tv544\""
Jul 29 01:16:51.495727 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.495704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.495822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.495738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7kr6\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-kube-api-access-b7kr6\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.597076 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.597014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvrq7\" (UniqueName: \"kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.597076 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.597049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.597247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.597078    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7kr6\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-kube-api-access-b7kr6\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.597247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.597126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.597247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.597197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.637501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.637471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7kr6\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-kube-api-access-b7kr6\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.643115 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.643092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd08f2e1-3998-4e82-8e68-004c2ea9cccf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nr2wr\" (UID: \"dd08f2e1-3998-4e82-8e68-004c2ea9cccf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:51.698271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.698247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.698360 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.698290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.698360 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.698328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvrq7\" (UniqueName: \"kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.698588 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.698570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.698622 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.698597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.714476 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.714447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvrq7\" (UniqueName: \"kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.745291 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.745268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:51.905692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.905657    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"]
Jul 29 01:16:51.908962 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:16:51.908935    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3adf272_6b3b_4c47_8ca5_d26823f44f90.slice/crio-8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42 WatchSource:0}: Error finding container 8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42: Status 404 returned error can't find the container with id 8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42
Jul 29 01:16:51.934951 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:51.934930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:52.083536 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:52.083503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nr2wr"]
Jul 29 01:16:52.085251 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:16:52.085220    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd08f2e1_3998_4e82_8e68_004c2ea9cccf.slice/crio-35ffe6b5cf954226a0ed5421960220ed45eaf0e243bb4a8ae1a658f5448d4d25 WatchSource:0}: Error finding container 35ffe6b5cf954226a0ed5421960220ed45eaf0e243bb4a8ae1a658f5448d4d25: Status 404 returned error can't find the container with id 35ffe6b5cf954226a0ed5421960220ed45eaf0e243bb4a8ae1a658f5448d4d25
Jul 29 01:16:52.313912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:52.313880    2564 generic.go:356] "Generic (PLEG): container finished" podID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerID="0abeb9774122bdae919c9de4b69e3f87dcdc69f071f44732f3a68b4e1986d6b1" exitCode=0
Jul 29 01:16:52.314077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:52.313970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72" event={"ID":"e3adf272-6b3b-4c47-8ca5-d26823f44f90","Type":"ContainerDied","Data":"0abeb9774122bdae919c9de4b69e3f87dcdc69f071f44732f3a68b4e1986d6b1"}
Jul 29 01:16:52.314077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:52.314035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72" event={"ID":"e3adf272-6b3b-4c47-8ca5-d26823f44f90","Type":"ContainerStarted","Data":"8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42"}
Jul 29 01:16:52.315212 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:52.315189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr" event={"ID":"dd08f2e1-3998-4e82-8e68-004c2ea9cccf","Type":"ContainerStarted","Data":"35ffe6b5cf954226a0ed5421960220ed45eaf0e243bb4a8ae1a658f5448d4d25"}
Jul 29 01:16:54.843892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:54.843854    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"]
Jul 29 01:16:54.846485 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:54.846464    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:54.849387 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:54.849363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ql74l\""
Jul 29 01:16:54.857497 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:54.857475    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"]
Jul 29 01:16:55.022841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.022801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpvgs\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-kube-api-access-dpvgs\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.023009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.022908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.123883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.123805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpvgs\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-kube-api-access-dpvgs\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.123883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.123871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.132896 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.132868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.133026 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.132971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpvgs\" (UniqueName: \"kubernetes.io/projected/1aeef3e8-9963-4d4e-a992-60e7a20a7c0b-kube-api-access-dpvgs\") pod \"cert-manager-cainjector-559cdd8c87-xsq9x\" (UID: \"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.158328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.158300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"
Jul 29 01:16:55.592546 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:55.592488    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x"]
Jul 29 01:16:55.595164 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:16:55.595129    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1aeef3e8_9963_4d4e_a992_60e7a20a7c0b.slice/crio-e72edeadcfd884eb8de077c377b049a3df7cdf4f0bc5e3c379378af0e61ab08e WatchSource:0}: Error finding container e72edeadcfd884eb8de077c377b049a3df7cdf4f0bc5e3c379378af0e61ab08e: Status 404 returned error can't find the container with id e72edeadcfd884eb8de077c377b049a3df7cdf4f0bc5e3c379378af0e61ab08e
Jul 29 01:16:56.335036 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.335003    2564 generic.go:356] "Generic (PLEG): container finished" podID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerID="cc8930f2d37663f51bf9ae323a46c54f615ce65706751defb3e53b83f140f59f" exitCode=0
Jul 29 01:16:56.335459 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.335075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72" event={"ID":"e3adf272-6b3b-4c47-8ca5-d26823f44f90","Type":"ContainerDied","Data":"cc8930f2d37663f51bf9ae323a46c54f615ce65706751defb3e53b83f140f59f"}
Jul 29 01:16:56.336614 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.336594    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x" event={"ID":"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b","Type":"ContainerStarted","Data":"cbf352bbea90250657c3065fef5dcf2829b1d13fc5ac41d3cef7b17dd1ba9411"}
Jul 29 01:16:56.336693 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.336621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x" event={"ID":"1aeef3e8-9963-4d4e-a992-60e7a20a7c0b","Type":"ContainerStarted","Data":"e72edeadcfd884eb8de077c377b049a3df7cdf4f0bc5e3c379378af0e61ab08e"}
Jul 29 01:16:56.338077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.338057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr" event={"ID":"dd08f2e1-3998-4e82-8e68-004c2ea9cccf","Type":"ContainerStarted","Data":"6b336253d4aea01142eb2fd61c4a29a906f96eaf4208aa5fee4b77c46d920f86"}
Jul 29 01:16:56.338784 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.338764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:16:56.377738 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.377693    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xsq9x" podStartSLOduration=2.377673592 podStartE2EDuration="2.377673592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:16:56.376192229 +0000 UTC m=+270.808074710" watchObservedRunningTime="2026-07-29 01:16:56.377673592 +0000 UTC m=+270.809556075"
Jul 29 01:16:56.392887 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:56.392847    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr" podStartSLOduration=2.019711982 podStartE2EDuration="5.392833044s" totalImagesPullingTime="3.373121062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:16:56.392418503 +0000 UTC m=+270.824300983" watchObservedRunningTime="2026-07-29 01:16:56.392833044 +0000 UTC m=+270.824715523"
Jul 29 01:16:57.344888 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:57.344856    2564 generic.go:356] "Generic (PLEG): container finished" podID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerID="eb6866cb5f4a3d880b098eb32d98ffb777868134be30ae98a9e1205b4c02b9a9" exitCode=0
Jul 29 01:16:57.345329 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:57.344925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72" event={"ID":"e3adf272-6b3b-4c47-8ca5-d26823f44f90","Type":"ContainerDied","Data":"eb6866cb5f4a3d880b098eb32d98ffb777868134be30ae98a9e1205b4c02b9a9"}
Jul 29 01:16:58.479631 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.479605    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:16:58.548711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.548681    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util\") pod \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") "
Jul 29 01:16:58.548863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.548728    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle\") pod \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") "
Jul 29 01:16:58.548863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.548758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvrq7\" (UniqueName: \"kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7\") pod \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\" (UID: \"e3adf272-6b3b-4c47-8ca5-d26823f44f90\") "
Jul 29 01:16:58.549102 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.549076    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle" (OuterVolumeSpecName: "bundle") pod "e3adf272-6b3b-4c47-8ca5-d26823f44f90" (UID: "e3adf272-6b3b-4c47-8ca5-d26823f44f90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:16:58.551012 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.550982    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7" (OuterVolumeSpecName: "kube-api-access-rvrq7") pod "e3adf272-6b3b-4c47-8ca5-d26823f44f90" (UID: "e3adf272-6b3b-4c47-8ca5-d26823f44f90"). InnerVolumeSpecName "kube-api-access-rvrq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:16:58.553096 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.553076    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util" (OuterVolumeSpecName: "util") pod "e3adf272-6b3b-4c47-8ca5-d26823f44f90" (UID: "e3adf272-6b3b-4c47-8ca5-d26823f44f90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:16:58.649510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.649438    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:58.649510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.649464    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvrq7\" (UniqueName: \"kubernetes.io/projected/e3adf272-6b3b-4c47-8ca5-d26823f44f90-kube-api-access-rvrq7\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:58.649510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:58.649476    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3adf272-6b3b-4c47-8ca5-d26823f44f90-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:16:59.355065 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:59.355028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72" event={"ID":"e3adf272-6b3b-4c47-8ca5-d26823f44f90","Type":"ContainerDied","Data":"8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42"}
Jul 29 01:16:59.355247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:59.355082    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d188df536d0825cb3c229c9f824501b9fd34efb2dbc9b1694e13845dff1ba42"
Jul 29 01:16:59.355247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:16:59.355035    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvmt72"
Jul 29 01:17:02.347449 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:02.347415    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nr2wr"
Jul 29 01:17:03.940158 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940111    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8rfl"]
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940403    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="extract"
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940414    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="extract"
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940425    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="pull"
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940432    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="pull"
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940445    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="util"
Jul 29 01:17:03.940464 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940450    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="util"
Jul 29 01:17:03.940639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.940500    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3adf272-6b3b-4c47-8ca5-d26823f44f90" containerName="extract"
Jul 29 01:17:03.942678 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.942660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:03.945247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.945224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7v749\""
Jul 29 01:17:03.952114 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.952091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8rfl"]
Jul 29 01:17:03.991510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.991487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-kube-api-access-brczk\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:03.991609 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:03.991532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.092396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.092342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-kube-api-access-brczk\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.092517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.092434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.100827 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.100805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.100978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.100960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brczk\" (UniqueName: \"kubernetes.io/projected/c1d3594f-5519-417a-9377-7356d0535090-kube-api-access-brczk\") pod \"cert-manager-55b4d954d5-n8rfl\" (UID: \"c1d3594f-5519-417a-9377-7356d0535090\") " pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.251256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.251187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n8rfl"
Jul 29 01:17:04.378255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:04.378230    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8rfl"]
Jul 29 01:17:04.380311 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:04.380285    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1d3594f_5519_417a_9377_7356d0535090.slice/crio-056db2011db3a18c1d270cb0ad6bac212a833d97f9efd4eda2119ea107092623 WatchSource:0}: Error finding container 056db2011db3a18c1d270cb0ad6bac212a833d97f9efd4eda2119ea107092623: Status 404 returned error can't find the container with id 056db2011db3a18c1d270cb0ad6bac212a833d97f9efd4eda2119ea107092623
Jul 29 01:17:05.385740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:05.385701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n8rfl" event={"ID":"c1d3594f-5519-417a-9377-7356d0535090","Type":"ContainerStarted","Data":"6cb15dde12009311a73d2e20274b96830207990ee3af4cabc80c310db4a7b3e3"}
Jul 29 01:17:05.386195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:05.386070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n8rfl" event={"ID":"c1d3594f-5519-417a-9377-7356d0535090","Type":"ContainerStarted","Data":"056db2011db3a18c1d270cb0ad6bac212a833d97f9efd4eda2119ea107092623"}
Jul 29 01:17:05.407625 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:05.407571    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n8rfl" podStartSLOduration=2.407553286 podStartE2EDuration="2.407553286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:05.406215945 +0000 UTC m=+279.838098409" watchObservedRunningTime="2026-07-29 01:17:05.407553286 +0000 UTC m=+279.839435767"
Jul 29 01:17:12.249356 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.249321    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"]
Jul 29 01:17:12.251916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.251901    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.259980 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.259962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 01:17:12.260158 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.260129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tv544\""
Jul 29 01:17:12.260213 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.260160    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 01:17:12.275178 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.275133    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"]
Jul 29 01:17:12.351807 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.351776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rwqr\" (UniqueName: \"kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.351930 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.351838    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.351930 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.351884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.452929 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.452904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rwqr\" (UniqueName: \"kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.453073 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.452951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.453073 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.452973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.453298 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.453277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.453357 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.453324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.467776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.467755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rwqr\" (UniqueName: \"kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.560869 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.560848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:12.696323 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:12.696249    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"]
Jul 29 01:17:12.699049 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:12.699019    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b358f3_1d81_4d45_96ca_06419ef0bdf8.slice/crio-3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717 WatchSource:0}: Error finding container 3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717: Status 404 returned error can't find the container with id 3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717
Jul 29 01:17:13.421048 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:13.421011    2564 generic.go:356] "Generic (PLEG): container finished" podID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerID="9035afe83726fdad49f4c9a129a47ab10784a2e91723db5c1fbdd2f863937dc9" exitCode=0
Jul 29 01:17:13.421485 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:13.421075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw" event={"ID":"17b358f3-1d81-4d45-96ca-06419ef0bdf8","Type":"ContainerDied","Data":"9035afe83726fdad49f4c9a129a47ab10784a2e91723db5c1fbdd2f863937dc9"}
Jul 29 01:17:13.421485 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:13.421131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw" event={"ID":"17b358f3-1d81-4d45-96ca-06419ef0bdf8","Type":"ContainerStarted","Data":"3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717"}
Jul 29 01:17:14.427422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:14.427383    2564 generic.go:356] "Generic (PLEG): container finished" podID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerID="8d54c92a3fac58473a26ec2c9224f4af09db63eee41fe9d1da22bd0278019ea2" exitCode=0
Jul 29 01:17:14.427759 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:14.427434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw" event={"ID":"17b358f3-1d81-4d45-96ca-06419ef0bdf8","Type":"ContainerDied","Data":"8d54c92a3fac58473a26ec2c9224f4af09db63eee41fe9d1da22bd0278019ea2"}
Jul 29 01:17:15.433739 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:15.433703    2564 generic.go:356] "Generic (PLEG): container finished" podID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerID="d287b0b0e3368c6647c5c46f96fc96dbf6199773d833c3ad39c97c9d73ac5d5d" exitCode=0
Jul 29 01:17:15.434104 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:15.433797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw" event={"ID":"17b358f3-1d81-4d45-96ca-06419ef0bdf8","Type":"ContainerDied","Data":"d287b0b0e3368c6647c5c46f96fc96dbf6199773d833c3ad39c97c9d73ac5d5d"}
Jul 29 01:17:16.560195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.560174    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:16.588162 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.588121    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rwqr\" (UniqueName: \"kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr\") pod \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") "
Jul 29 01:17:16.588266 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.588184    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle\") pod \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") "
Jul 29 01:17:16.588266 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.588209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util\") pod \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\" (UID: \"17b358f3-1d81-4d45-96ca-06419ef0bdf8\") "
Jul 29 01:17:16.588867 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.588845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle" (OuterVolumeSpecName: "bundle") pod "17b358f3-1d81-4d45-96ca-06419ef0bdf8" (UID: "17b358f3-1d81-4d45-96ca-06419ef0bdf8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:17:16.590187 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.590163    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr" (OuterVolumeSpecName: "kube-api-access-9rwqr") pod "17b358f3-1d81-4d45-96ca-06419ef0bdf8" (UID: "17b358f3-1d81-4d45-96ca-06419ef0bdf8"). InnerVolumeSpecName "kube-api-access-9rwqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:17:16.594787 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.594761    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util" (OuterVolumeSpecName: "util") pod "17b358f3-1d81-4d45-96ca-06419ef0bdf8" (UID: "17b358f3-1d81-4d45-96ca-06419ef0bdf8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:17:16.688863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.688779    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rwqr\" (UniqueName: \"kubernetes.io/projected/17b358f3-1d81-4d45-96ca-06419ef0bdf8-kube-api-access-9rwqr\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:16.688863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.688814    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:16.688863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:16.688830    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17b358f3-1d81-4d45-96ca-06419ef0bdf8-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:17.445328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:17.445295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw" event={"ID":"17b358f3-1d81-4d45-96ca-06419ef0bdf8","Type":"ContainerDied","Data":"3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717"}
Jul 29 01:17:17.445481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:17.445344    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c0f7c00fc80f489a05402c92ebd84a71952fa0c22235d36df5fd5261a51c717"
Jul 29 01:17:17.445481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:17.445306    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdmghw"
Jul 29 01:17:25.964769 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:25.964743    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:17:25.969303 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:25.969282    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:17:25.976066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:25.976047    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 01:17:28.238811 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.238777    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"]
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239065    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="extract"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239076    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="extract"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239086    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="util"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239092    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="util"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239101    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="pull"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239106    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="pull"
Jul 29 01:17:28.240590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.239188    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17b358f3-1d81-4d45-96ca-06419ef0bdf8" containerName="extract"
Jul 29 01:17:28.241270 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.241249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.244423 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.244391    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 01:17:28.244589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.244422    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9ksqq\""
Jul 29 01:17:28.244677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.244486    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 01:17:28.244737 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.244683    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 01:17:28.244737 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.244521    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 01:17:28.262038 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.262013    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"]
Jul 29 01:17:28.380972 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.380941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.381125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.380981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-tmp\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.381125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.381008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf57m\" (UniqueName: \"kubernetes.io/projected/063f8783-bc85-45aa-93a9-935de65404af-kube-api-access-bf57m\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.381125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.381103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-manifests\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.381314 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.381185    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.482453 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.482424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.482613 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.482477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-tmp\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.482613 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.482516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf57m\" (UniqueName: \"kubernetes.io/projected/063f8783-bc85-45aa-93a9-935de65404af-kube-api-access-bf57m\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.482613 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.482541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-manifests\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.482613 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.482571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.483048 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.483026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-manifests\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.484847 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.484827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063f8783-bc85-45aa-93a9-935de65404af-tmp\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.484991 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.484975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.485177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.485139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063f8783-bc85-45aa-93a9-935de65404af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.491448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.491388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf57m\" (UniqueName: \"kubernetes.io/projected/063f8783-bc85-45aa-93a9-935de65404af-kube-api-access-bf57m\") pod \"opendatahub-operator-controller-manager-6f677d9b6b-847s4\" (UID: \"063f8783-bc85-45aa-93a9-935de65404af\") " pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.551902 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.551884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:28.692520 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.692491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"]
Jul 29 01:17:28.696139 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:28.696107    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063f8783_bc85_45aa_93a9_935de65404af.slice/crio-3ebf7765f47248e1ec72ac182610b884d7bd0fb3dfa9727073f58ce29f0d5281 WatchSource:0}: Error finding container 3ebf7765f47248e1ec72ac182610b884d7bd0fb3dfa9727073f58ce29f0d5281: Status 404 returned error can't find the container with id 3ebf7765f47248e1ec72ac182610b884d7bd0fb3dfa9727073f58ce29f0d5281
Jul 29 01:17:28.698046 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:28.698028    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:17:29.509188 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:29.509134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4" event={"ID":"063f8783-bc85-45aa-93a9-935de65404af","Type":"ContainerStarted","Data":"3ebf7765f47248e1ec72ac182610b884d7bd0fb3dfa9727073f58ce29f0d5281"}
Jul 29 01:17:31.523721 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:31.523694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4" event={"ID":"063f8783-bc85-45aa-93a9-935de65404af","Type":"ContainerStarted","Data":"b59d0964a71e6de191b5d61a2b301185c10e2857bbef047b89e75123a4ca4554"}
Jul 29 01:17:33.534377 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:33.534341    2564 generic.go:356] "Generic (PLEG): container finished" podID="063f8783-bc85-45aa-93a9-935de65404af" containerID="b59d0964a71e6de191b5d61a2b301185c10e2857bbef047b89e75123a4ca4554" exitCode=0
Jul 29 01:17:33.534749 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:33.534414    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4" event={"ID":"063f8783-bc85-45aa-93a9-935de65404af","Type":"ContainerDied","Data":"b59d0964a71e6de191b5d61a2b301185c10e2857bbef047b89e75123a4ca4554"}
Jul 29 01:17:34.540998 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:34.540961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4" event={"ID":"063f8783-bc85-45aa-93a9-935de65404af","Type":"ContainerStarted","Data":"db07cb44e1e6a00e3f019ba141b66deb31efc407da8e437983dd4004d8d87774"}
Jul 29 01:17:34.541397 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:34.541187    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:34.564583 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:34.564540    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4" podStartSLOduration=2.460939545 podStartE2EDuration="6.564526592s" totalImagesPullingTime="3.043646581s" totalInitContainerRuntime="1.059940466s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:34.562187696 +0000 UTC m=+308.994070175" watchObservedRunningTime="2026-07-29 01:17:34.564526592 +0000 UTC m=+308.996409072"
Jul 29 01:17:45.548352 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:45.548322    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f677d9b6b-847s4"
Jul 29 01:17:47.882525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.882486    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"]
Jul 29 01:17:47.885777 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.885755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:47.888972 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.888937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 01:17:47.889112 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.888979    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fxk7q\""
Jul 29 01:17:47.890248 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.890232    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 01:17:47.913816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:47.913788    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"]
Jul 29 01:17:48.034116 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.034090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b853d0b7-acd5-4b42-bbd4-86109856d87c-tmp\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.034252 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.034128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5bm\" (UniqueName: \"kubernetes.io/projected/b853d0b7-acd5-4b42-bbd4-86109856d87c-kube-api-access-jc5bm\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.034252 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.034173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b853d0b7-acd5-4b42-bbd4-86109856d87c-tls-certs\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.135068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.134995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b853d0b7-acd5-4b42-bbd4-86109856d87c-tmp\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.135068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.135032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5bm\" (UniqueName: \"kubernetes.io/projected/b853d0b7-acd5-4b42-bbd4-86109856d87c-kube-api-access-jc5bm\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.135068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.135051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b853d0b7-acd5-4b42-bbd4-86109856d87c-tls-certs\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.137333 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.137313    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b853d0b7-acd5-4b42-bbd4-86109856d87c-tmp\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.137542 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.137525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b853d0b7-acd5-4b42-bbd4-86109856d87c-tls-certs\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.157234 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.157207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5bm\" (UniqueName: \"kubernetes.io/projected/b853d0b7-acd5-4b42-bbd4-86109856d87c-kube-api-access-jc5bm\") pod \"kube-auth-proxy-5dfd7865d-2jsdx\" (UID: \"b853d0b7-acd5-4b42-bbd4-86109856d87c\") " pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.187519 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.187496    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wwvrd"]
Jul 29 01:17:48.190580 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.190565    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.195675 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.195659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"
Jul 29 01:17:48.196904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.196811    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 01:17:48.197050 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.197027    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tmhwq\""
Jul 29 01:17:48.207472 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.207451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wwvrd"]
Jul 29 01:17:48.327080 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.327057    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx"]
Jul 29 01:17:48.329103 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:48.329078    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb853d0b7_acd5_4b42_bbd4_86109856d87c.slice/crio-e6a23469cf66c58bb9f69a576a31ce6a804e77503c4f2439264e11bc0a2a06cc WatchSource:0}: Error finding container e6a23469cf66c58bb9f69a576a31ce6a804e77503c4f2439264e11bc0a2a06cc: Status 404 returned error can't find the container with id e6a23469cf66c58bb9f69a576a31ce6a804e77503c4f2439264e11bc0a2a06cc
Jul 29 01:17:48.337633 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.337613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.337713 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.337669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsvxt\" (UniqueName: \"kubernetes.io/projected/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-kube-api-access-wsvxt\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.438193 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.438122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvxt\" (UniqueName: \"kubernetes.io/projected/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-kube-api-access-wsvxt\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.438300 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.438215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.438353 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:17:48.438331    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 01:17:48.438403 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:17:48.438392    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert podName:8eb18c7f-3ea8-4ded-a295-23c4f841faf7 nodeName:}" failed. No retries permitted until 2026-07-29 01:17:48.938372376 +0000 UTC m=+323.370254837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert") pod "odh-model-controller-7998c67dc4-wwvrd" (UID: "8eb18c7f-3ea8-4ded-a295-23c4f841faf7") : secret "odh-model-controller-webhook-cert" not found
Jul 29 01:17:48.451747 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.451724    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvxt\" (UniqueName: \"kubernetes.io/projected/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-kube-api-access-wsvxt\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.603224 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.603186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx" event={"ID":"b853d0b7-acd5-4b42-bbd4-86109856d87c","Type":"ContainerStarted","Data":"e6a23469cf66c58bb9f69a576a31ce6a804e77503c4f2439264e11bc0a2a06cc"}
Jul 29 01:17:48.943013 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.942982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:48.945489 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:48.945464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8eb18c7f-3ea8-4ded-a295-23c4f841faf7-cert\") pod \"odh-model-controller-7998c67dc4-wwvrd\" (UID: \"8eb18c7f-3ea8-4ded-a295-23c4f841faf7\") " pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:49.100234 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.100198    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:49.124129 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.124092    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-742jf"]
Jul 29 01:17:49.127826 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.127805    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.131116 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.131089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 01:17:49.145855 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.145820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-742jf"]
Jul 29 01:17:49.245002 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.244975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb41e47-c95f-4d77-bc73-1480a6eec141-tls-certs\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.245161 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.245011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n59x\" (UniqueName: \"kubernetes.io/projected/9bb41e47-c95f-4d77-bc73-1480a6eec141-kube-api-access-8n59x\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.264031 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.263998    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wwvrd"]
Jul 29 01:17:49.267265 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:49.267237    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb18c7f_3ea8_4ded_a295_23c4f841faf7.slice/crio-fdf369737b7998aa88bb1ddc470bff7a488d84abd102ee8fa8489cfb98c43379 WatchSource:0}: Error finding container fdf369737b7998aa88bb1ddc470bff7a488d84abd102ee8fa8489cfb98c43379: Status 404 returned error can't find the container with id fdf369737b7998aa88bb1ddc470bff7a488d84abd102ee8fa8489cfb98c43379
Jul 29 01:17:49.345821 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.345796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb41e47-c95f-4d77-bc73-1480a6eec141-tls-certs\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.345952 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.345829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n59x\" (UniqueName: \"kubernetes.io/projected/9bb41e47-c95f-4d77-bc73-1480a6eec141-kube-api-access-8n59x\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.348711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.348686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb41e47-c95f-4d77-bc73-1480a6eec141-tls-certs\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.356867 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.356826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n59x\" (UniqueName: \"kubernetes.io/projected/9bb41e47-c95f-4d77-bc73-1480a6eec141-kube-api-access-8n59x\") pod \"model-serving-api-54d89bd5fc-742jf\" (UID: \"9bb41e47-c95f-4d77-bc73-1480a6eec141\") " pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.360646 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.360623    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:17:49.364021 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.363999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.366859 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.366837    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 01:17:49.367098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.367022    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 01:17:49.367197 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.367095    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7jm8k\""
Jul 29 01:17:49.367510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.367490    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 01:17:49.367737 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.367715    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 01:17:49.386664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.386641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:17:49.446971 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.446945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.446985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.447006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447217 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.447092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447217 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.447141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447217 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.447181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xbt\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.447342 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.447250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.455957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.455899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:49.548351 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548404    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.548659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.548571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xbt\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.549646 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.549618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.551565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.551450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.551670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.551572    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.551670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.551645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.551776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.551685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.557875 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.557831    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.558589 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.558561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xbt\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt\") pod \"istiod-openshift-gateway-7b7995d5cd-hw9nf\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:49.610020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.609986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd" event={"ID":"8eb18c7f-3ea8-4ded-a295-23c4f841faf7","Type":"ContainerStarted","Data":"fdf369737b7998aa88bb1ddc470bff7a488d84abd102ee8fa8489cfb98c43379"}
Jul 29 01:17:49.610610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.610589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-742jf"]
Jul 29 01:17:49.675515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:49.675485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:50.220892 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:50.220861    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb41e47_c95f_4d77_bc73_1480a6eec141.slice/crio-852be155257550b9adf50be24990e8f2b06903e81b8bc48d1944e9c960413285 WatchSource:0}: Error finding container 852be155257550b9adf50be24990e8f2b06903e81b8bc48d1944e9c960413285: Status 404 returned error can't find the container with id 852be155257550b9adf50be24990e8f2b06903e81b8bc48d1944e9c960413285
Jul 29 01:17:50.368033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:50.367934    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:17:50.372131 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:50.372098    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ead6e5c_2640_4f3c_8111_b6e7ccc49dff.slice/crio-f0c40a55b8baec3b84ec3b5655700dc26a27e7fc87bb895732445dcb2d20bf14 WatchSource:0}: Error finding container f0c40a55b8baec3b84ec3b5655700dc26a27e7fc87bb895732445dcb2d20bf14: Status 404 returned error can't find the container with id f0c40a55b8baec3b84ec3b5655700dc26a27e7fc87bb895732445dcb2d20bf14
Jul 29 01:17:50.620805 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:50.620748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx" event={"ID":"b853d0b7-acd5-4b42-bbd4-86109856d87c","Type":"ContainerStarted","Data":"904affd7120f630a135a9077eae76373f4cda66f421b82e91b1c21ed26505462"}
Jul 29 01:17:50.623093 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:50.623048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-742jf" event={"ID":"9bb41e47-c95f-4d77-bc73-1480a6eec141","Type":"ContainerStarted","Data":"852be155257550b9adf50be24990e8f2b06903e81b8bc48d1944e9c960413285"}
Jul 29 01:17:50.625116 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:50.625079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" event={"ID":"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff","Type":"ContainerStarted","Data":"f0c40a55b8baec3b84ec3b5655700dc26a27e7fc87bb895732445dcb2d20bf14"}
Jul 29 01:17:50.646787 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:50.646736    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dfd7865d-2jsdx" podStartSLOduration=1.698628023 podStartE2EDuration="3.646716353s" totalImagesPullingTime="1.94808833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:50.643213471 +0000 UTC m=+325.075095953" watchObservedRunningTime="2026-07-29 01:17:50.646716353 +0000 UTC m=+325.078598833"
Jul 29 01:17:53.643709 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.643654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd" event={"ID":"8eb18c7f-3ea8-4ded-a295-23c4f841faf7","Type":"ContainerStarted","Data":"f82f597a7cf5976b3b4217052b5b1e9ae3958a3032f0dc7285f72ab94140ca16"}
Jul 29 01:17:53.644195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.643722    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:17:53.645285 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.645254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-742jf" event={"ID":"9bb41e47-c95f-4d77-bc73-1480a6eec141","Type":"ContainerStarted","Data":"d85f31778435f1e320c3efee11d0a305f975d3d13d1823633a41ec2327644593"}
Jul 29 01:17:53.645412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.645375    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:17:53.665348 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.665305    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd" podStartSLOduration=2.147216185 podStartE2EDuration="5.665291584s" totalImagesPullingTime="3.518075399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:53.662775132 +0000 UTC m=+328.094657613" watchObservedRunningTime="2026-07-29 01:17:53.665291584 +0000 UTC m=+328.097174063"
Jul 29 01:17:53.680385 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:53.680203    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-742jf" podStartSLOduration=2.120295822 podStartE2EDuration="4.680186808s" totalImagesPullingTime="2.559890986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:53.680119878 +0000 UTC m=+328.112002357" watchObservedRunningTime="2026-07-29 01:17:53.680186808 +0000 UTC m=+328.112069289"
Jul 29 01:17:54.377056 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:54.377018    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:17:54.377181 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:54.377088    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:17:54.651979 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:54.651895    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" event={"ID":"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff","Type":"ContainerStarted","Data":"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"}
Jul 29 01:17:54.652395 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:54.652087    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:54.687668 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:54.687612    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" podStartSLOduration=1.686056953 podStartE2EDuration="5.687594428s" totalImagesPullingTime="4.001537475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:54.683852195 +0000 UTC m=+329.115734678" watchObservedRunningTime="2026-07-29 01:17:54.687594428 +0000 UTC m=+329.119476910"
Jul 29 01:17:55.658312 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:55.658278    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-hw9nf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 01:17:55.658658 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:55.658342    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 01:17:56.064045 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.063998    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"]
Jul 29 01:17:56.067866 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.067848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.073252 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.073227    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7tdc7\""
Jul 29 01:17:56.116161 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.116132    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"]
Jul 29 01:17:56.206930 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.206902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207052 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.206946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207052 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.206979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46dc861d-630b-4031-9da7-e0a245c3d436-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207141 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207198 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207242 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjq2s\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-kube-api-access-gjq2s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207281 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207318 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46dc861d-630b-4031-9da7-e0a245c3d436-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.207451 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.207432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.308683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46dc861d-630b-4031-9da7-e0a245c3d436-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.308816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308695    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.308816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.308816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjq2s\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-kube-api-access-gjq2s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.308816 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46dc861d-630b-4031-9da7-e0a245c3d436-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.308967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309300 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.309184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309350 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.309326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309387 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.309354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309427 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.309396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.309525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.309505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46dc861d-630b-4031-9da7-e0a245c3d436-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.310889 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.310866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46dc861d-630b-4031-9da7-e0a245c3d436-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.311242 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.311223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46dc861d-630b-4031-9da7-e0a245c3d436-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.321018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.320968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.321730 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.321713    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjq2s\" (UniqueName: \"kubernetes.io/projected/46dc861d-630b-4031-9da7-e0a245c3d436-kube-api-access-gjq2s\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d\" (UID: \"46dc861d-630b-4031-9da7-e0a245c3d436\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.378395 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.378365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:17:56.552329 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.552296    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"]
Jul 29 01:17:56.552889 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:56.552862    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46dc861d_630b_4031_9da7_e0a245c3d436.slice/crio-cec5f08bdfd28f04312db677fb136c3ed59db31f10e41a171ad126368b5280ab WatchSource:0}: Error finding container cec5f08bdfd28f04312db677fb136c3ed59db31f10e41a171ad126368b5280ab: Status 404 returned error can't find the container with id cec5f08bdfd28f04312db677fb136c3ed59db31f10e41a171ad126368b5280ab
Jul 29 01:17:56.662638 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:56.662570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d" event={"ID":"46dc861d-630b-4031-9da7-e0a245c3d436","Type":"ContainerStarted","Data":"cec5f08bdfd28f04312db677fb136c3ed59db31f10e41a171ad126368b5280ab"}
Jul 29 01:17:58.384948 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.384913    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"]
Jul 29 01:17:58.388756 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.388736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.391958 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.391939    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 01:17:58.392646 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.392506    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fl9rv\""
Jul 29 01:17:58.399593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.399573    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"]
Jul 29 01:17:58.402990 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.402969    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.406993 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.406954    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 01:17:58.407102 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.406993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-s6kfv\""
Jul 29 01:17:58.410363 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.410343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"]
Jul 29 01:17:58.422025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.421999    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"]
Jul 29 01:17:58.529027 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.528994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.529205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.529032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppl92\" (UniqueName: \"kubernetes.io/projected/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-kube-api-access-ppl92\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.529205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.529107    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg6b8\" (UniqueName: \"kubernetes.io/projected/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-kube-api-access-dg6b8\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.529205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.529200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-cert\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.630514 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.630471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-cert\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.630694 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.630542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.630694 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.630574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppl92\" (UniqueName: \"kubernetes.io/projected/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-kube-api-access-ppl92\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.630694 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.630619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg6b8\" (UniqueName: \"kubernetes.io/projected/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-kube-api-access-dg6b8\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.634738 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.634161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-cert\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.634738 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.634610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.641554 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.641494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg6b8\" (UniqueName: \"kubernetes.io/projected/8f4551ce-4708-4db3-aec3-9b3a3acd75c9-kube-api-access-dg6b8\") pod \"kserve-controller-manager-d6f4b65b8-trmp8\" (UID: \"8f4551ce-4708-4db3-aec3-9b3a3acd75c9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.645294 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.645263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppl92\" (UniqueName: \"kubernetes.io/projected/b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680-kube-api-access-ppl92\") pod \"llmisvc-controller-manager-777c6bd7d5-pzbhb\" (UID: \"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:58.657841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.657807    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:58.702015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.701996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:17:58.713010 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:58.712990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:17:59.257142 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.257106    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:17:59.257342 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.257315    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerName="discovery" containerID="cri-o://7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0" gracePeriod=30
Jul 29 01:17:59.526579 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.525834    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:17:59.526579 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.525941    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:17:59.526579 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.526005    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:17:59.665063 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.664901    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:59.684822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.684792    2564 generic.go:356] "Generic (PLEG): container finished" podID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerID="7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0" exitCode=0
Jul 29 01:17:59.684964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.684836    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"
Jul 29 01:17:59.684964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.684871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" event={"ID":"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff","Type":"ContainerDied","Data":"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"}
Jul 29 01:17:59.684964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.684917    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf" event={"ID":"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff","Type":"ContainerDied","Data":"f0c40a55b8baec3b84ec3b5655700dc26a27e7fc87bb895732445dcb2d20bf14"}
Jul 29 01:17:59.684964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.684944    2564 scope.go:117] "RemoveContainer" containerID="7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"
Jul 29 01:17:59.686675 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.686644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d" event={"ID":"46dc861d-630b-4031-9da7-e0a245c3d436","Type":"ContainerStarted","Data":"3c732b40faef46d26e739f792cb861e5116d9357602fb13e423a7184f64fe1a8"}
Jul 29 01:17:59.694429 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.694409    2564 scope.go:117] "RemoveContainer" containerID="7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"
Jul 29 01:17:59.694719 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:17:59.694697    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0\": container with ID starting with 7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0 not found: ID does not exist" containerID="7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"
Jul 29 01:17:59.694768 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.694732    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0"} err="failed to get container status \"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0\": rpc error: code = NotFound desc = could not find container \"7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0\": container with ID starting with 7b0a8730ac7bad163135c3b2c3f893435ff21b5789e731094ecfef8ccb3577e0 not found: ID does not exist"
Jul 29 01:17:59.758938 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.758823    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d" podStartSLOduration=0.788525169 podStartE2EDuration="3.758802702s" totalImagesPullingTime="2.970277533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:17:59.757093766 +0000 UTC m=+334.188976247" watchObservedRunningTime="2026-07-29 01:17:59.758802702 +0000 UTC m=+334.190685182"
Jul 29 01:17:59.841508 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841473    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2xbt\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841537    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841595    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841648    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841886 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841674    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.841886 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.841720    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig\") pod \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\" (UID: \"3ead6e5c-2640-4f3c-8111-b6e7ccc49dff\") "
Jul 29 01:17:59.842642 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.842123    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:17:59.844520 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844480    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:17:59.844520 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844509    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs" (OuterVolumeSpecName: "local-certs") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:17:59.844674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844580    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts" (OuterVolumeSpecName: "cacerts") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:17:59.844768 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844745    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt" (OuterVolumeSpecName: "kube-api-access-g2xbt") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "kube-api-access-g2xbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:17:59.844880 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844862    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:17:59.844988 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.844973    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" (UID: "3ead6e5c-2640-4f3c-8111-b6e7ccc49dff"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:17:59.883345 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:59.883316    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4551ce_4708_4db3_aec3_9b3a3acd75c9.slice/crio-d64fc2bf70b75ea70ded14363b679cb9657f23b10f5f184a4851141620f51dc9 WatchSource:0}: Error finding container d64fc2bf70b75ea70ded14363b679cb9657f23b10f5f184a4851141620f51dc9: Status 404 returned error can't find the container with id d64fc2bf70b75ea70ded14363b679cb9657f23b10f5f184a4851141620f51dc9
Jul 29 01:17:59.888689 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:17:59.888664    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb1d47a29_93d7_4c7a_ba8a_1cb63a3f0680.slice/crio-a17c712ebd96993e101f55a889b5e92199f52a6ccc550142bfd2c3fee1801041 WatchSource:0}: Error finding container a17c712ebd96993e101f55a889b5e92199f52a6ccc550142bfd2c3fee1801041: Status 404 returned error can't find the container with id a17c712ebd96993e101f55a889b5e92199f52a6ccc550142bfd2c3fee1801041
Jul 29 01:17:59.892770 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.892746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"]
Jul 29 01:17:59.894436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.894418    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"]
Jul 29 01:17:59.942757 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942729    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2xbt\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-kube-api-access-g2xbt\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942758    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-local-certs\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942773    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-token\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942786    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-ca-configmap\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942800    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-csr-dns-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942817    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-cacerts\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:17:59.942872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:17:59.942830    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff-istio-kubeconfig\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:18:00.013812 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.013783    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:18:00.064936 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.064908    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-hw9nf"]
Jul 29 01:18:00.121215 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.118399    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" path="/var/lib/kubelet/pods/3ead6e5c-2640-4f3c-8111-b6e7ccc49dff/volumes"
Jul 29 01:18:00.379570 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.379498    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:18:00.384928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.384903    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:18:00.696880 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.696787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8" event={"ID":"8f4551ce-4708-4db3-aec3-9b3a3acd75c9","Type":"ContainerStarted","Data":"d64fc2bf70b75ea70ded14363b679cb9657f23b10f5f184a4851141620f51dc9"}
Jul 29 01:18:00.699223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.699089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb" event={"ID":"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680","Type":"ContainerStarted","Data":"a17c712ebd96993e101f55a889b5e92199f52a6ccc550142bfd2c3fee1801041"}
Jul 29 01:18:00.699532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.699478    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:18:00.700493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:00.700464    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d"
Jul 29 01:18:03.717926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.717890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb" event={"ID":"b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680","Type":"ContainerStarted","Data":"4072f006f992bc8e6563d116206579fe9fb251fe896c7a4106377fca10ec9f52"}
Jul 29 01:18:03.718475 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.718048    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:18:03.719493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.719469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8" event={"ID":"8f4551ce-4708-4db3-aec3-9b3a3acd75c9","Type":"ContainerStarted","Data":"6a0ea618afc21088a767e14082d279932c50bd19dc05eac14245b632d8d6a90c"}
Jul 29 01:18:03.719621 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.719606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:18:03.739205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.739158    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb" podStartSLOduration=2.732802887 podStartE2EDuration="5.739127222s" totalImagesPullingTime="3.006324335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:18:03.737468342 +0000 UTC m=+338.169350824" watchObservedRunningTime="2026-07-29 01:18:03.739127222 +0000 UTC m=+338.171009706"
Jul 29 01:18:03.756451 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:03.756416    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8" podStartSLOduration=2.750756269 podStartE2EDuration="5.756403297s" totalImagesPullingTime="3.005647028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:18:03.755739577 +0000 UTC m=+338.187622060" watchObservedRunningTime="2026-07-29 01:18:03.756403297 +0000 UTC m=+338.188285777"
Jul 29 01:18:04.654011 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:04.653979    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wwvrd"
Jul 29 01:18:04.655977 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:04.655957    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-742jf"
Jul 29 01:18:34.728218 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:34.728185    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pzbhb"
Jul 29 01:18:34.731248 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:34.731221    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-trmp8"
Jul 29 01:18:48.851268 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.851228    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:48.851683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.851570    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerName="discovery"
Jul 29 01:18:48.851683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.851581    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerName="discovery"
Jul 29 01:18:48.851683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.851639    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ead6e5c-2640-4f3c-8111-b6e7ccc49dff" containerName="discovery"
Jul 29 01:18:48.853579 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.853561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:48.858193 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.858140    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-s2gs9\""
Jul 29 01:18:48.858819 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.858796    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 01:18:48.860536 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.860495    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 01:18:48.876659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.876635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:48.919667 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:48.919646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv6f9\" (UniqueName: \"kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9\") pod \"kuadrant-operator-catalog-6fp8w\" (UID: \"12e0a8ff-57d6-479b-b89f-c098596a3e54\") " pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:49.020379 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.020354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv6f9\" (UniqueName: \"kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9\") pod \"kuadrant-operator-catalog-6fp8w\" (UID: \"12e0a8ff-57d6-479b-b89f-c098596a3e54\") " pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:49.029053 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.029025    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv6f9\" (UniqueName: \"kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9\") pod \"kuadrant-operator-catalog-6fp8w\" (UID: \"12e0a8ff-57d6-479b-b89f-c098596a3e54\") " pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:49.163706 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.163644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:49.191533 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.191508    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:49.294795 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.294773    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:49.296836 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:18:49.296812    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e0a8ff_57d6_479b_b89f_c098596a3e54.slice/crio-05c1546dd5127b4044cfa2a56e9eaeeaea65c4a9e41eb16c3b082ca6adef9de9 WatchSource:0}: Error finding container 05c1546dd5127b4044cfa2a56e9eaeeaea65c4a9e41eb16c3b082ca6adef9de9: Status 404 returned error can't find the container with id 05c1546dd5127b4044cfa2a56e9eaeeaea65c4a9e41eb16c3b082ca6adef9de9
Jul 29 01:18:49.400188 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.400164    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8f6f"]
Jul 29 01:18:49.402967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.402951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:49.412481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.412454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8f6f"]
Jul 29 01:18:49.524679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.524660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97vdf\" (UniqueName: \"kubernetes.io/projected/a91d336a-4ecf-4483-977e-7eef09289749-kube-api-access-97vdf\") pod \"kuadrant-operator-catalog-k8f6f\" (UID: \"a91d336a-4ecf-4483-977e-7eef09289749\") " pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:49.625324 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.625297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97vdf\" (UniqueName: \"kubernetes.io/projected/a91d336a-4ecf-4483-977e-7eef09289749-kube-api-access-97vdf\") pod \"kuadrant-operator-catalog-k8f6f\" (UID: \"a91d336a-4ecf-4483-977e-7eef09289749\") " pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:49.637329 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.637298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97vdf\" (UniqueName: \"kubernetes.io/projected/a91d336a-4ecf-4483-977e-7eef09289749-kube-api-access-97vdf\") pod \"kuadrant-operator-catalog-k8f6f\" (UID: \"a91d336a-4ecf-4483-977e-7eef09289749\") " pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:49.712493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.712472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:49.858320 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.858292    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k8f6f"]
Jul 29 01:18:49.871420 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:18:49.871384    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91d336a_4ecf_4483_977e_7eef09289749.slice/crio-fdda2c643f591a469157e6208e6296406bbfd078b92c7654abec80b58315f6fe WatchSource:0}: Error finding container fdda2c643f591a469157e6208e6296406bbfd078b92c7654abec80b58315f6fe: Status 404 returned error can't find the container with id fdda2c643f591a469157e6208e6296406bbfd078b92c7654abec80b58315f6fe
Jul 29 01:18:49.949343 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.949298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f" event={"ID":"a91d336a-4ecf-4483-977e-7eef09289749","Type":"ContainerStarted","Data":"fdda2c643f591a469157e6208e6296406bbfd078b92c7654abec80b58315f6fe"}
Jul 29 01:18:49.950739 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:49.950697    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" event={"ID":"12e0a8ff-57d6-479b-b89f-c098596a3e54","Type":"ContainerStarted","Data":"05c1546dd5127b4044cfa2a56e9eaeeaea65c4a9e41eb16c3b082ca6adef9de9"}
Jul 29 01:18:51.962684 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:51.962642    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f" event={"ID":"a91d336a-4ecf-4483-977e-7eef09289749","Type":"ContainerStarted","Data":"0339b5bfd088c90dcb4aeb7eae7579db5aa1fe95c798af45ebe32395ce5fd9eb"}
Jul 29 01:18:51.964132 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:51.964102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" event={"ID":"12e0a8ff-57d6-479b-b89f-c098596a3e54","Type":"ContainerStarted","Data":"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"}
Jul 29 01:18:51.964292 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:51.964112    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" podUID="12e0a8ff-57d6-479b-b89f-c098596a3e54" containerName="registry-server" containerID="cri-o://ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45" gracePeriod=2
Jul 29 01:18:51.978239 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:51.978187    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f" podStartSLOduration=1.6719979980000002 podStartE2EDuration="2.978170159s" totalImagesPullingTime="1.306172161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:18:51.977770907 +0000 UTC m=+386.409653387" watchObservedRunningTime="2026-07-29 01:18:51.978170159 +0000 UTC m=+386.410052640"
Jul 29 01:18:51.996175 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:51.996122    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" podStartSLOduration=2.1159848 podStartE2EDuration="3.996108729s" totalImagesPullingTime="1.880123929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:18:51.993684601 +0000 UTC m=+386.425567075" watchObservedRunningTime="2026-07-29 01:18:51.996108729 +0000 UTC m=+386.427991208"
Jul 29 01:18:52.207280 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.207255    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:52.350204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.350179    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv6f9\" (UniqueName: \"kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9\") pod \"12e0a8ff-57d6-479b-b89f-c098596a3e54\" (UID: \"12e0a8ff-57d6-479b-b89f-c098596a3e54\") "
Jul 29 01:18:52.352341 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.352306    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9" (OuterVolumeSpecName: "kube-api-access-gv6f9") pod "12e0a8ff-57d6-479b-b89f-c098596a3e54" (UID: "12e0a8ff-57d6-479b-b89f-c098596a3e54"). InnerVolumeSpecName "kube-api-access-gv6f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:18:52.451514 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.451481    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv6f9\" (UniqueName: \"kubernetes.io/projected/12e0a8ff-57d6-479b-b89f-c098596a3e54-kube-api-access-gv6f9\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:18:52.970372 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.970337    2564 generic.go:356] "Generic (PLEG): container finished" podID="12e0a8ff-57d6-479b-b89f-c098596a3e54" containerID="ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45" exitCode=0
Jul 29 01:18:52.970805 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.970392    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w"
Jul 29 01:18:52.970805 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.970425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" event={"ID":"12e0a8ff-57d6-479b-b89f-c098596a3e54","Type":"ContainerDied","Data":"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"}
Jul 29 01:18:52.970805 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.970520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6fp8w" event={"ID":"12e0a8ff-57d6-479b-b89f-c098596a3e54","Type":"ContainerDied","Data":"05c1546dd5127b4044cfa2a56e9eaeeaea65c4a9e41eb16c3b082ca6adef9de9"}
Jul 29 01:18:52.970805 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.970550    2564 scope.go:117] "RemoveContainer" containerID="ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"
Jul 29 01:18:52.981974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.981955    2564 scope.go:117] "RemoveContainer" containerID="ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"
Jul 29 01:18:52.982257 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:18:52.982236    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45\": container with ID starting with ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45 not found: ID does not exist" containerID="ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"
Jul 29 01:18:52.982325 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.982266    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45"} err="failed to get container status \"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45\": rpc error: code = NotFound desc = could not find container \"ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45\": container with ID starting with ecf32ec417b4e76c3fc8aab444bb1169b9b9ab2a28a2c9af56a66cfa7b75fc45 not found: ID does not exist"
Jul 29 01:18:52.994036 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.994013    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:52.996569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:52.996550    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6fp8w"]
Jul 29 01:18:54.116243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:54.116212    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e0a8ff-57d6-479b-b89f-c098596a3e54" path="/var/lib/kubelet/pods/12e0a8ff-57d6-479b-b89f-c098596a3e54/volumes"
Jul 29 01:18:59.712991 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:59.712952    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:59.713506 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:59.713015    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:18:59.735741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:18:59.735718    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:19:00.026385 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:00.026359    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k8f6f"
Jul 29 01:19:04.846075 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.846034    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"]
Jul 29 01:19:04.846929 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.846903    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e0a8ff-57d6-479b-b89f-c098596a3e54" containerName="registry-server"
Jul 29 01:19:04.847017 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.846931    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e0a8ff-57d6-479b-b89f-c098596a3e54" containerName="registry-server"
Jul 29 01:19:04.847102 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.847090    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e0a8ff-57d6-479b-b89f-c098596a3e54" containerName="registry-server"
Jul 29 01:19:04.852978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.852952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:04.856003 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.855978    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t5xr4\""
Jul 29 01:19:04.857314 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.857291    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"]
Jul 29 01:19:04.944857 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.944832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:04.944959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.944884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:04.944959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:04.944912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5879r\" (UniqueName: \"kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.045726 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.045701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.045850 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.045737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5879r\" (UniqueName: \"kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.045850 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.045770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.046100 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.046079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.046189 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.046108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.054735 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.054715    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5879r\" (UniqueName: \"kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.163411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.163346    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:05.313357 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.313333    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"]
Jul 29 01:19:05.315089 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:05.315055    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b0b85a_588f_4b60_9dac_df9aac75b967.slice/crio-2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b WatchSource:0}: Error finding container 2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b: Status 404 returned error can't find the container with id 2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b
Jul 29 01:19:05.448451 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.448427    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"]
Jul 29 01:19:05.451994 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.451979    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.461535 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.461514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"]
Jul 29 01:19:05.549713 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.549687    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhrxj\" (UniqueName: \"kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.549821 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.549731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.549821 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.549764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.650692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.650668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhrxj\" (UniqueName: \"kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.650779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.650707    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.650779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.650734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.651044 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.651030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.651122 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.651104    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.659782 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.659758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhrxj\" (UniqueName: \"kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.787618 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.787597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:05.846279 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.846253    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"]
Jul 29 01:19:05.851138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.851112    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:05.863810 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.863784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"]
Jul 29 01:19:05.920022 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.919996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"]
Jul 29 01:19:05.921442 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:05.921418    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2274627_6f17_4c41_83fd_355627aa9a46.slice/crio-d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d WatchSource:0}: Error finding container d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d: Status 404 returned error can't find the container with id d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d
Jul 29 01:19:05.952996 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.952971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:05.953123 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.953012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:05.953123 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:05.953079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcvns\" (UniqueName: \"kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.036045 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.036018    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2274627-6f17-4c41-83fd-355627aa9a46" containerID="2f93ee44e2c4eabab9d1906604e5605c0fb53286efb47f0e7e8213f6dd67bf51" exitCode=0
Jul 29 01:19:06.036140 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.036098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6" event={"ID":"d2274627-6f17-4c41-83fd-355627aa9a46","Type":"ContainerDied","Data":"2f93ee44e2c4eabab9d1906604e5605c0fb53286efb47f0e7e8213f6dd67bf51"}
Jul 29 01:19:06.036215 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.036168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6" event={"ID":"d2274627-6f17-4c41-83fd-355627aa9a46","Type":"ContainerStarted","Data":"d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d"}
Jul 29 01:19:06.037735 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.037685    2564 generic.go:356] "Generic (PLEG): container finished" podID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerID="bc3fdf909eee1893abfd346a4a335fd6905ce7bdb39cb2bbf6e7ec85857dc3db" exitCode=0
Jul 29 01:19:06.037821 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.037755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx" event={"ID":"85b0b85a-588f-4b60-9dac-df9aac75b967","Type":"ContainerDied","Data":"bc3fdf909eee1893abfd346a4a335fd6905ce7bdb39cb2bbf6e7ec85857dc3db"}
Jul 29 01:19:06.037821 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.037798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx" event={"ID":"85b0b85a-588f-4b60-9dac-df9aac75b967","Type":"ContainerStarted","Data":"2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b"}
Jul 29 01:19:06.054125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.054103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.054243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.054163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.054243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.054204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcvns\" (UniqueName: \"kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.054482 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.054440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.054610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.054586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.067394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.067375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcvns\" (UniqueName: \"kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.160939 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.160916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:06.246675 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.246642    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"]
Jul 29 01:19:06.251894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.251871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.258961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.258906    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"]
Jul 29 01:19:06.290920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.290862    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"]
Jul 29 01:19:06.293694 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:06.293663    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f9dc99_fa81_4270_8c7a_6fc877f3fee9.slice/crio-d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658 WatchSource:0}: Error finding container d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658: Status 404 returned error can't find the container with id d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658
Jul 29 01:19:06.357667 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.357647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg457\" (UniqueName: \"kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.357763 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.357690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.357851 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.357822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.458934 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.458908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg457\" (UniqueName: \"kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.459031 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.458958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.459031 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.459024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.459352 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.459334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.459397 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.459353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.467950 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.467931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg457\" (UniqueName: \"kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.565290 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.565226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:06.859270 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.859243    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"]
Jul 29 01:19:06.880571 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.880549    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67c8645ddf-ghq55"]
Jul 29 01:19:06.885804 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.885783    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:06.897062 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:06.897035    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4f6887_e9a5_4e3e_b5a7_8f6c70ee644b.slice/crio-ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b WatchSource:0}: Error finding container ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b: Status 404 returned error can't find the container with id ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b
Jul 29 01:19:06.916559 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:06.916456    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c8645ddf-ghq55"]
Jul 29 01:19:07.044824 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.044798    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2274627-6f17-4c41-83fd-355627aa9a46" containerID="bfca10deccee98d72c6fadc96f752bf5e196eb6e23e86d5f85b16d82e0d5309b" exitCode=0
Jul 29 01:19:07.044934 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.044883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6" event={"ID":"d2274627-6f17-4c41-83fd-355627aa9a46","Type":"ContainerDied","Data":"bfca10deccee98d72c6fadc96f752bf5e196eb6e23e86d5f85b16d82e0d5309b"}
Jul 29 01:19:07.046495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.046472    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerID="708197cff81e8c818cce51ad99a58dcb353433bebb3d74188ccd2c73b7b09eee" exitCode=0
Jul 29 01:19:07.046572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.046512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds" event={"ID":"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b","Type":"ContainerDied","Data":"708197cff81e8c818cce51ad99a58dcb353433bebb3d74188ccd2c73b7b09eee"}
Jul 29 01:19:07.046572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.046558    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds" event={"ID":"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b","Type":"ContainerStarted","Data":"ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b"}
Jul 29 01:19:07.048042 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.048023    2564 generic.go:356] "Generic (PLEG): container finished" podID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerID="e6048746ee528f63b3c47a31e2fd9ac8714036e9ef9f65f460263fd99f70e665" exitCode=0
Jul 29 01:19:07.048131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.048101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv" event={"ID":"26f9dc99-fa81-4270-8c7a-6fc877f3fee9","Type":"ContainerDied","Data":"e6048746ee528f63b3c47a31e2fd9ac8714036e9ef9f65f460263fd99f70e665"}
Jul 29 01:19:07.048243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.048139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv" event={"ID":"26f9dc99-fa81-4270-8c7a-6fc877f3fee9","Type":"ContainerStarted","Data":"d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658"}
Jul 29 01:19:07.050192 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.050168    2564 generic.go:356] "Generic (PLEG): container finished" podID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerID="cdee6723a0e26a97e4906b07facc9cbd8f14be94e07f4b83beccb00ab9c0fba6" exitCode=0
Jul 29 01:19:07.050286 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.050202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx" event={"ID":"85b0b85a-588f-4b60-9dac-df9aac75b967","Type":"ContainerDied","Data":"cdee6723a0e26a97e4906b07facc9cbd8f14be94e07f4b83beccb00ab9c0fba6"}
Jul 29 01:19:07.064534 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-console-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-service-ca\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-oauth-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064844 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlbt\" (UniqueName: \"kubernetes.io/projected/e6f3700e-38d0-441d-895b-c903b5682f60-kube-api-access-cnlbt\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064844 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-oauth-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.064844 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.064786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-trusted-ca-bundle\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166124 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-oauth-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166136    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-trusted-ca-bundle\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166323 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-console-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166323 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-service-ca\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-oauth-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.166423 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.166390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlbt\" (UniqueName: \"kubernetes.io/projected/e6f3700e-38d0-441d-895b-c903b5682f60-kube-api-access-cnlbt\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.167382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.167355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-service-ca\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.167491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.167402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-trusted-ca-bundle\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.167757 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.167734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-oauth-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.167969 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.167949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6f3700e-38d0-441d-895b-c903b5682f60-console-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.169996 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.169972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-oauth-config\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.170664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.170643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6f3700e-38d0-441d-895b-c903b5682f60-console-serving-cert\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.187445 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.187397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlbt\" (UniqueName: \"kubernetes.io/projected/e6f3700e-38d0-441d-895b-c903b5682f60-kube-api-access-cnlbt\") pod \"console-67c8645ddf-ghq55\" (UID: \"e6f3700e-38d0-441d-895b-c903b5682f60\") " pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.205256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.205239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:07.346563 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:07.346538    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67c8645ddf-ghq55"]
Jul 29 01:19:07.348981 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:07.348949    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f3700e_38d0_441d_895b_c903b5682f60.slice/crio-d28466edb4779531d054a2f175f7162aa25368e0a3b1d1d09af4325ddaf3e4c6 WatchSource:0}: Error finding container d28466edb4779531d054a2f175f7162aa25368e0a3b1d1d09af4325ddaf3e4c6: Status 404 returned error can't find the container with id d28466edb4779531d054a2f175f7162aa25368e0a3b1d1d09af4325ddaf3e4c6
Jul 29 01:19:08.057504 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.057480    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2274627-6f17-4c41-83fd-355627aa9a46" containerID="e9a0f889ee0dd01d8ea57532a8f992e2d8e2e0aa698ad12f28332fdef0d98ca3" exitCode=0
Jul 29 01:19:08.057874 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.057569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6" event={"ID":"d2274627-6f17-4c41-83fd-355627aa9a46","Type":"ContainerDied","Data":"e9a0f889ee0dd01d8ea57532a8f992e2d8e2e0aa698ad12f28332fdef0d98ca3"}
Jul 29 01:19:08.059401 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.059376    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerID="dfcb41aa0e36e2fd2fc8150b146a033abd1e2f5da72f8d78b8e0f03417dc3f32" exitCode=0
Jul 29 01:19:08.059501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.059437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds" event={"ID":"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b","Type":"ContainerDied","Data":"dfcb41aa0e36e2fd2fc8150b146a033abd1e2f5da72f8d78b8e0f03417dc3f32"}
Jul 29 01:19:08.061104 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.061078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c8645ddf-ghq55" event={"ID":"e6f3700e-38d0-441d-895b-c903b5682f60","Type":"ContainerStarted","Data":"9dc06163589e57521d8c2f45c562e3d4bf66091d3e06d45d0e6d5ef00d78e389"}
Jul 29 01:19:08.061207 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.061130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67c8645ddf-ghq55" event={"ID":"e6f3700e-38d0-441d-895b-c903b5682f60","Type":"ContainerStarted","Data":"d28466edb4779531d054a2f175f7162aa25368e0a3b1d1d09af4325ddaf3e4c6"}
Jul 29 01:19:08.062915 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.062896    2564 generic.go:356] "Generic (PLEG): container finished" podID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerID="2058cd7b8c098cae92d9e84e32fd8a485a432ad272739b18262e577a02fa0592" exitCode=0
Jul 29 01:19:08.063002 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.062971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv" event={"ID":"26f9dc99-fa81-4270-8c7a-6fc877f3fee9","Type":"ContainerDied","Data":"2058cd7b8c098cae92d9e84e32fd8a485a432ad272739b18262e577a02fa0592"}
Jul 29 01:19:08.065303 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.065281    2564 generic.go:356] "Generic (PLEG): container finished" podID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerID="45024cb236bbf0b08378d574599dd78d90fe7d2a7d507f668196d0b84bd85197" exitCode=0
Jul 29 01:19:08.065394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.065313    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx" event={"ID":"85b0b85a-588f-4b60-9dac-df9aac75b967","Type":"ContainerDied","Data":"45024cb236bbf0b08378d574599dd78d90fe7d2a7d507f668196d0b84bd85197"}
Jul 29 01:19:08.138485 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:08.137595    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67c8645ddf-ghq55" podStartSLOduration=2.137572792 podStartE2EDuration="2.137572792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:08.134655394 +0000 UTC m=+402.566537875" watchObservedRunningTime="2026-07-29 01:19:08.137572792 +0000 UTC m=+402.569455272"
Jul 29 01:19:09.076174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.076126    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerID="a7f481a92cf34e70fff1efa649e11325c81eca95e23ee7dcfa55630d5f75eae1" exitCode=0
Jul 29 01:19:09.076526 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.076189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds" event={"ID":"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b","Type":"ContainerDied","Data":"a7f481a92cf34e70fff1efa649e11325c81eca95e23ee7dcfa55630d5f75eae1"}
Jul 29 01:19:09.078237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.078204    2564 generic.go:356] "Generic (PLEG): container finished" podID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerID="6db50896ea5e10126449856c0b21e0aae7450cdc10699bcc7519066a9f515ef5" exitCode=0
Jul 29 01:19:09.078352 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.078281    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv" event={"ID":"26f9dc99-fa81-4270-8c7a-6fc877f3fee9","Type":"ContainerDied","Data":"6db50896ea5e10126449856c0b21e0aae7450cdc10699bcc7519066a9f515ef5"}
Jul 29 01:19:09.214301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.214279    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:09.242755 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.242736    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:09.384394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle\") pod \"85b0b85a-588f-4b60-9dac-df9aac75b967\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") "
Jul 29 01:19:09.384394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384369    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util\") pod \"85b0b85a-588f-4b60-9dac-df9aac75b967\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") "
Jul 29 01:19:09.384603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384399    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5879r\" (UniqueName: \"kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r\") pod \"85b0b85a-588f-4b60-9dac-df9aac75b967\" (UID: \"85b0b85a-588f-4b60-9dac-df9aac75b967\") "
Jul 29 01:19:09.384603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384426    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhrxj\" (UniqueName: \"kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj\") pod \"d2274627-6f17-4c41-83fd-355627aa9a46\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") "
Jul 29 01:19:09.384603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384451    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle\") pod \"d2274627-6f17-4c41-83fd-355627aa9a46\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") "
Jul 29 01:19:09.384757 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384627    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util\") pod \"d2274627-6f17-4c41-83fd-355627aa9a46\" (UID: \"d2274627-6f17-4c41-83fd-355627aa9a46\") "
Jul 29 01:19:09.385022 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.384990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle" (OuterVolumeSpecName: "bundle") pod "85b0b85a-588f-4b60-9dac-df9aac75b967" (UID: "85b0b85a-588f-4b60-9dac-df9aac75b967"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:09.385164 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.385132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle" (OuterVolumeSpecName: "bundle") pod "d2274627-6f17-4c41-83fd-355627aa9a46" (UID: "d2274627-6f17-4c41-83fd-355627aa9a46"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:09.386603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.386582    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r" (OuterVolumeSpecName: "kube-api-access-5879r") pod "85b0b85a-588f-4b60-9dac-df9aac75b967" (UID: "85b0b85a-588f-4b60-9dac-df9aac75b967"). InnerVolumeSpecName "kube-api-access-5879r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:09.386867 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.386851    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj" (OuterVolumeSpecName: "kube-api-access-nhrxj") pod "d2274627-6f17-4c41-83fd-355627aa9a46" (UID: "d2274627-6f17-4c41-83fd-355627aa9a46"). InnerVolumeSpecName "kube-api-access-nhrxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:09.390814 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.390793    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util" (OuterVolumeSpecName: "util") pod "85b0b85a-588f-4b60-9dac-df9aac75b967" (UID: "85b0b85a-588f-4b60-9dac-df9aac75b967"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:09.391284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.391265    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util" (OuterVolumeSpecName: "util") pod "d2274627-6f17-4c41-83fd-355627aa9a46" (UID: "d2274627-6f17-4c41-83fd-355627aa9a46"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:09.485370 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485342    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5879r\" (UniqueName: \"kubernetes.io/projected/85b0b85a-588f-4b60-9dac-df9aac75b967-kube-api-access-5879r\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:09.485370 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485368    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhrxj\" (UniqueName: \"kubernetes.io/projected/d2274627-6f17-4c41-83fd-355627aa9a46-kube-api-access-nhrxj\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:09.485560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485379    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:09.485560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485393    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2274627-6f17-4c41-83fd-355627aa9a46-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:09.485560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485402    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:09.485560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:09.485409    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85b0b85a-588f-4b60-9dac-df9aac75b967-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.085853 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.085821    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx"
Jul 29 01:19:10.085853 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.085835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx" event={"ID":"85b0b85a-588f-4b60-9dac-df9aac75b967","Type":"ContainerDied","Data":"2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b"}
Jul 29 01:19:10.086284 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.085882    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2299ce06c90749407242276272bb68d705238310442e6ecc431211c94a50d39b"
Jul 29 01:19:10.087596 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.087575    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6"
Jul 29 01:19:10.087596 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.087587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6" event={"ID":"d2274627-6f17-4c41-83fd-355627aa9a46","Type":"ContainerDied","Data":"d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d"}
Jul 29 01:19:10.087744 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.087622    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d98821eeeb84822c2350a38486a2cb21d611244eb2c0a12dcc9d75197465500d"
Jul 29 01:19:10.246531 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.246509    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:10.248604 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.248589    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:10.393064 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.392978    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcvns\" (UniqueName: \"kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns\") pod \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") "
Jul 29 01:19:10.393064 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393023    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle\") pod \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") "
Jul 29 01:19:10.393064 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393060    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util\") pod \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") "
Jul 29 01:19:10.393353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393077    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg457\" (UniqueName: \"kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457\") pod \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") "
Jul 29 01:19:10.393353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle\") pod \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\" (UID: \"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b\") "
Jul 29 01:19:10.393353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393170    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util\") pod \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\" (UID: \"26f9dc99-fa81-4270-8c7a-6fc877f3fee9\") "
Jul 29 01:19:10.393668 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393636    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle" (OuterVolumeSpecName: "bundle") pod "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" (UID: "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:10.393817 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.393796    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle" (OuterVolumeSpecName: "bundle") pod "26f9dc99-fa81-4270-8c7a-6fc877f3fee9" (UID: "26f9dc99-fa81-4270-8c7a-6fc877f3fee9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:10.395275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.395159    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns" (OuterVolumeSpecName: "kube-api-access-hcvns") pod "26f9dc99-fa81-4270-8c7a-6fc877f3fee9" (UID: "26f9dc99-fa81-4270-8c7a-6fc877f3fee9"). InnerVolumeSpecName "kube-api-access-hcvns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:10.395658 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.395628    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457" (OuterVolumeSpecName: "kube-api-access-xg457") pod "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" (UID: "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b"). InnerVolumeSpecName "kube-api-access-xg457". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:10.399054 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.399033    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util" (OuterVolumeSpecName: "util") pod "26f9dc99-fa81-4270-8c7a-6fc877f3fee9" (UID: "26f9dc99-fa81-4270-8c7a-6fc877f3fee9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:10.399522 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.399502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util" (OuterVolumeSpecName: "util") pod "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" (UID: "0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:10.494228 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494201    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.494228 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494221    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg457\" (UniqueName: \"kubernetes.io/projected/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-kube-api-access-xg457\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.494353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494240    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.494353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494249    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-util\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.494353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494258    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcvns\" (UniqueName: \"kubernetes.io/projected/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-kube-api-access-hcvns\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:10.494353 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:10.494266    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26f9dc99-fa81-4270-8c7a-6fc877f3fee9-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:11.094930 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.094905    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds"
Jul 29 01:19:11.094930 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.094920    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds" event={"ID":"0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b","Type":"ContainerDied","Data":"ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b"}
Jul 29 01:19:11.095475 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.094966    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef9b5c4f84200efdb635547f130f73f6fe8b13b2d37fcbbd1b0ee3f01f379b8b"
Jul 29 01:19:11.096872 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.096850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv" event={"ID":"26f9dc99-fa81-4270-8c7a-6fc877f3fee9","Type":"ContainerDied","Data":"d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658"}
Jul 29 01:19:11.096961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.096889    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1d45fa0dd2de3ff45b048cb83d462f49affbf3616c12ecea1c19c590605d658"
Jul 29 01:19:11.096961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:11.096904    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv"
Jul 29 01:19:16.349741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.349705    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"]
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350048    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="util"
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350058    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="util"
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350067    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="util"
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350073    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="util"
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350080    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="pull"
Jul 29 01:19:16.350084 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350085    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350096    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350101    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350108    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350113    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350128    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350133    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350140    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350160    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350170    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350178    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350200    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350208    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="extract"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350216    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350221    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="pull"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350238    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="util"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350243    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="util"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350253    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="util"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350257    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="util"
Jul 29 01:19:16.350313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350317    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2274627-6f17-4c41-83fd-355627aa9a46" containerName="extract"
Jul 29 01:19:16.350822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350327    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b0b85a-588f-4b60-9dac-df9aac75b967" containerName="extract"
Jul 29 01:19:16.350822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350333    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="26f9dc99-fa81-4270-8c7a-6fc877f3fee9" containerName="extract"
Jul 29 01:19:16.350822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.350340    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b" containerName="extract"
Jul 29 01:19:16.354588 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.354566    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:16.357740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.357725    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lvmw7\""
Jul 29 01:19:16.370522 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.370502    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"]
Jul 29 01:19:16.547167 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.543526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5vqm\" (UniqueName: \"kubernetes.io/projected/bbe7a322-69fe-4aed-ae5f-8a12f85aa46f-kube-api-access-w5vqm\") pod \"authorino-operator-75cc8d5cb7-ppw52\" (UID: \"bbe7a322-69fe-4aed-ae5f-8a12f85aa46f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:16.645020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.644944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5vqm\" (UniqueName: \"kubernetes.io/projected/bbe7a322-69fe-4aed-ae5f-8a12f85aa46f-kube-api-access-w5vqm\") pod \"authorino-operator-75cc8d5cb7-ppw52\" (UID: \"bbe7a322-69fe-4aed-ae5f-8a12f85aa46f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:16.659225 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.659200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5vqm\" (UniqueName: \"kubernetes.io/projected/bbe7a322-69fe-4aed-ae5f-8a12f85aa46f-kube-api-access-w5vqm\") pod \"authorino-operator-75cc8d5cb7-ppw52\" (UID: \"bbe7a322-69fe-4aed-ae5f-8a12f85aa46f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:16.665204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.665178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:16.801383 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:16.801347    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"]
Jul 29 01:19:16.803044 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:16.803016    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe7a322_69fe_4aed_ae5f_8a12f85aa46f.slice/crio-4c72148b8ab3d688bca4dda664766970ba5519ba438f19edad99f52ed3fdb312 WatchSource:0}: Error finding container 4c72148b8ab3d688bca4dda664766970ba5519ba438f19edad99f52ed3fdb312: Status 404 returned error can't find the container with id 4c72148b8ab3d688bca4dda664766970ba5519ba438f19edad99f52ed3fdb312
Jul 29 01:19:17.128741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:17.128713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52" event={"ID":"bbe7a322-69fe-4aed-ae5f-8a12f85aa46f","Type":"ContainerStarted","Data":"4c72148b8ab3d688bca4dda664766970ba5519ba438f19edad99f52ed3fdb312"}
Jul 29 01:19:17.206247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:17.206213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:17.206373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:17.206267    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:17.210852 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:17.210833    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:18.139485 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:18.139455    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67c8645ddf-ghq55"
Jul 29 01:19:18.225346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:18.225309    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:19:19.141724 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.141698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52" event={"ID":"bbe7a322-69fe-4aed-ae5f-8a12f85aa46f","Type":"ContainerStarted","Data":"99a9b4b4a06b66445eb4c4cce8134db4f948884c252f55c2b7cfc7d5831a33ab"}
Jul 29 01:19:19.166058 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.166010    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52" podStartSLOduration=0.905369253 podStartE2EDuration="3.165990034s" totalImagesPullingTime="2.260620781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:19.162924802 +0000 UTC m=+413.594807283" watchObservedRunningTime="2026-07-29 01:19:19.165990034 +0000 UTC m=+413.597872516"
Jul 29 01:19:19.909039 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.909006    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"]
Jul 29 01:19:19.912317 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.912301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:19.915416 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.915393    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 01:19:19.915600 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.915451    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jghls\""
Jul 29 01:19:19.926702 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.926680    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"]
Jul 29 01:19:19.972871 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:19.972844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6vjd\" (UniqueName: \"kubernetes.io/projected/23061558-82b7-4b1f-9de2-e18beb0c4bf8-kube-api-access-g6vjd\") pod \"dns-operator-controller-manager-864dd78c4c-nkj79\" (UID: \"23061558-82b7-4b1f-9de2-e18beb0c4bf8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:20.073566 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:20.073537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6vjd\" (UniqueName: \"kubernetes.io/projected/23061558-82b7-4b1f-9de2-e18beb0c4bf8-kube-api-access-g6vjd\") pod \"dns-operator-controller-manager-864dd78c4c-nkj79\" (UID: \"23061558-82b7-4b1f-9de2-e18beb0c4bf8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:20.084528 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:20.084508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6vjd\" (UniqueName: \"kubernetes.io/projected/23061558-82b7-4b1f-9de2-e18beb0c4bf8-kube-api-access-g6vjd\") pod \"dns-operator-controller-manager-864dd78c4c-nkj79\" (UID: \"23061558-82b7-4b1f-9de2-e18beb0c4bf8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:20.146648 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:20.146626    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:20.222494 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:20.222441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:20.359424 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:20.359398    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"]
Jul 29 01:19:20.360770 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:20.360743    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23061558_82b7_4b1f_9de2_e18beb0c4bf8.slice/crio-3d0cd731df36af2543baaf57e5b92ae4fed859d47037e06097b7f97b0fc7dfba WatchSource:0}: Error finding container 3d0cd731df36af2543baaf57e5b92ae4fed859d47037e06097b7f97b0fc7dfba: Status 404 returned error can't find the container with id 3d0cd731df36af2543baaf57e5b92ae4fed859d47037e06097b7f97b0fc7dfba
Jul 29 01:19:21.153841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:21.153805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79" event={"ID":"23061558-82b7-4b1f-9de2-e18beb0c4bf8","Type":"ContainerStarted","Data":"3d0cd731df36af2543baaf57e5b92ae4fed859d47037e06097b7f97b0fc7dfba"}
Jul 29 01:19:23.166916 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.166883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79" event={"ID":"23061558-82b7-4b1f-9de2-e18beb0c4bf8","Type":"ContainerStarted","Data":"52916325fd9ce313c3781ef0d5701d61a4036edb0c14d8bb5313ae0bbce6a1b6"}
Jul 29 01:19:23.167301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.166939    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:23.196969 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.196916    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79" podStartSLOduration=2.338194604 podStartE2EDuration="4.196900467s" totalImagesPullingTime="1.858705863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:23.195564581 +0000 UTC m=+417.627447078" watchObservedRunningTime="2026-07-29 01:19:23.196900467 +0000 UTC m=+417.628782947"
Jul 29 01:19:23.272734 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.272698    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"]
Jul 29 01:19:23.276478 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.276457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:23.280885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.280865    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2vflp\""
Jul 29 01:19:23.300495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.300467    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"]
Jul 29 01:19:23.397540 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.397510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch7t9\" (UniqueName: \"kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9\") pod \"limitador-operator-controller-manager-859d7b78bd-gjphm\" (UID: \"77b9b2cb-2954-46a0-a972-7a288dc3119a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:23.498687 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.498585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch7t9\" (UniqueName: \"kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9\") pod \"limitador-operator-controller-manager-859d7b78bd-gjphm\" (UID: \"77b9b2cb-2954-46a0-a972-7a288dc3119a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:23.515599 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.515567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch7t9\" (UniqueName: \"kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9\") pod \"limitador-operator-controller-manager-859d7b78bd-gjphm\" (UID: \"77b9b2cb-2954-46a0-a972-7a288dc3119a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:23.586892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.586858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:23.746561 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:23.746526    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"]
Jul 29 01:19:23.750388 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:23.750319    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b9b2cb_2954_46a0_a972_7a288dc3119a.slice/crio-7a70937e180eafe601ab64f76b9a834acca1035aebad7d31f8597ced0a1a850f WatchSource:0}: Error finding container 7a70937e180eafe601ab64f76b9a834acca1035aebad7d31f8597ced0a1a850f: Status 404 returned error can't find the container with id 7a70937e180eafe601ab64f76b9a834acca1035aebad7d31f8597ced0a1a850f
Jul 29 01:19:24.173072 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:24.173037    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm" event={"ID":"77b9b2cb-2954-46a0-a972-7a288dc3119a","Type":"ContainerStarted","Data":"7a70937e180eafe601ab64f76b9a834acca1035aebad7d31f8597ced0a1a850f"}
Jul 29 01:19:26.185635 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:26.185601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm" event={"ID":"77b9b2cb-2954-46a0-a972-7a288dc3119a","Type":"ContainerStarted","Data":"bdf39cc5da7a94f1fc95e9c8247eead64f5f968f6dfb9daef0b9f6045b3425fb"}
Jul 29 01:19:26.186001 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:26.185700    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:26.206198 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:26.206158    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm" podStartSLOduration=1.637506427 podStartE2EDuration="3.206131048s" totalImagesPullingTime="1.568624621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:26.205885765 +0000 UTC m=+420.637768245" watchObservedRunningTime="2026-07-29 01:19:26.206131048 +0000 UTC m=+420.638013527"
Jul 29 01:19:27.382721 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.382684    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"]
Jul 29 01:19:27.387639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.387616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.392493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.392467    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-77d5k\""
Jul 29 01:19:27.416366 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.416341    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"]
Jul 29 01:19:27.428142 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.428116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dvr\" (UniqueName: \"kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.428268 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.428244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.529131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.529106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.529244 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.529195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dvr\" (UniqueName: \"kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.529541 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.529521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.543425 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.543400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dvr\" (UniqueName: \"kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.697332 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.697272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:27.835280 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:27.835254    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"]
Jul 29 01:19:27.837873 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:27.837849    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86d50ba_1036_4e00_b505_3693ae3157b5.slice/crio-378af86dc7b4c44d57b0d01761dde5d8f0f636c96565f18169a3f8b32b15c7bd WatchSource:0}: Error finding container 378af86dc7b4c44d57b0d01761dde5d8f0f636c96565f18169a3f8b32b15c7bd: Status 404 returned error can't find the container with id 378af86dc7b4c44d57b0d01761dde5d8f0f636c96565f18169a3f8b32b15c7bd
Jul 29 01:19:28.198312 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:28.198282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk" event={"ID":"e86d50ba-1036-4e00-b505-3693ae3157b5","Type":"ContainerStarted","Data":"378af86dc7b4c44d57b0d01761dde5d8f0f636c96565f18169a3f8b32b15c7bd"}
Jul 29 01:19:31.155995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:31.155965    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ppw52"
Jul 29 01:19:32.223195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:32.223136    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk" event={"ID":"e86d50ba-1036-4e00-b505-3693ae3157b5","Type":"ContainerStarted","Data":"b585bb908ba0074add486579818fbfa001c400cd57c78489ef9df52fe15e78b5"}
Jul 29 01:19:32.223595 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:32.223330    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:32.263481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:32.263429    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk" podStartSLOduration=1.918619552 podStartE2EDuration="5.263411248s" totalImagesPullingTime="3.344791696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:32.261799008 +0000 UTC m=+426.693681506" watchObservedRunningTime="2026-07-29 01:19:32.263411248 +0000 UTC m=+426.695293731"
Jul 29 01:19:34.175354 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:34.175328    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nkj79"
Jul 29 01:19:37.193693 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:37.193666    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:43.231345 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.231312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:43.246321 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.246261    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b69cdc7fc-4jlps" podUID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" containerName="console" containerID="cri-o://d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d" gracePeriod=15
Jul 29 01:19:43.501729 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.501706    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:19:43.562003 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.561967    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562171 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562171 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562308 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562189    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9hgv\" (UniqueName: \"kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562308 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562308 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562250    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562308 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562297    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config\") pod \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\" (UID: \"842952c0-4de9-4ac3-9d58-bca1a4cfca51\") "
Jul 29 01:19:43.562490 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:19:43.562576 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562548    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:19:43.562640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562579    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca" (OuterVolumeSpecName: "service-ca") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:19:43.562696 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562663    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config" (OuterVolumeSpecName: "console-config") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:19:43.562753 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562736    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-config\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.562794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562754    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-trusted-ca-bundle\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.562794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562764    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.562794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.562772    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/842952c0-4de9-4ac3-9d58-bca1a4cfca51-oauth-serving-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.564434 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.564402    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:19:43.564532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.564454    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:19:43.564532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.564470    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv" (OuterVolumeSpecName: "kube-api-access-h9hgv") pod "842952c0-4de9-4ac3-9d58-bca1a4cfca51" (UID: "842952c0-4de9-4ac3-9d58-bca1a4cfca51"). InnerVolumeSpecName "kube-api-access-h9hgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:43.663894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.663857    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9hgv\" (UniqueName: \"kubernetes.io/projected/842952c0-4de9-4ac3-9d58-bca1a4cfca51-kube-api-access-h9hgv\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.663894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.663889    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-serving-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:43.663894 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:43.663902    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/842952c0-4de9-4ac3-9d58-bca1a4cfca51-console-oauth-config\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:44.290533 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.290494    2564 generic.go:356] "Generic (PLEG): container finished" podID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" containerID="d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d" exitCode=0
Jul 29 01:19:44.290904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.290537    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69cdc7fc-4jlps"
Jul 29 01:19:44.290904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.290575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69cdc7fc-4jlps" event={"ID":"842952c0-4de9-4ac3-9d58-bca1a4cfca51","Type":"ContainerDied","Data":"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"}
Jul 29 01:19:44.290904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.290625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69cdc7fc-4jlps" event={"ID":"842952c0-4de9-4ac3-9d58-bca1a4cfca51","Type":"ContainerDied","Data":"e7fe444394e9cffe4c588d0323a3d564802e2038dbec4b7db2256930ac754b2d"}
Jul 29 01:19:44.290904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.290645    2564 scope.go:117] "RemoveContainer" containerID="d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"
Jul 29 01:19:44.298897 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.298881    2564 scope.go:117] "RemoveContainer" containerID="d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"
Jul 29 01:19:44.299158 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:44.299127    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d\": container with ID starting with d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d not found: ID does not exist" containerID="d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"
Jul 29 01:19:44.299227 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.299169    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d"} err="failed to get container status \"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d\": rpc error: code = NotFound desc = could not find container \"d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d\": container with ID starting with d1def32653d684d9c04507ffe640208afc4465540fc3c88bb7b50e6202ec5e6d not found: ID does not exist"
Jul 29 01:19:44.334376 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.334352    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:19:44.338729 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:44.338708    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b69cdc7fc-4jlps"]
Jul 29 01:19:45.060570 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.060537    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"]
Jul 29 01:19:45.060726 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.060705    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" containerName="manager" containerID="cri-o://b585bb908ba0074add486579818fbfa001c400cd57c78489ef9df52fe15e78b5" gracePeriod=2
Jul 29 01:19:45.071891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.071864    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"]
Jul 29 01:19:45.112572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.112543    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"]
Jul 29 01:19:45.112775 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.112754    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" containerName="manager" containerID="cri-o://bdf39cc5da7a94f1fc95e9c8247eead64f5f968f6dfb9daef0b9f6045b3425fb" gracePeriod=2
Jul 29 01:19:45.115611 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.115576    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.134066 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.134039    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"]
Jul 29 01:19:45.148416 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148388    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:19:45.148934 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148919    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" containerName="console"
Jul 29 01:19:45.149019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148937    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" containerName="console"
Jul 29 01:19:45.149019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148953    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" containerName="manager"
Jul 29 01:19:45.149019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148962    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" containerName="manager"
Jul 29 01:19:45.149019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148973    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" containerName="manager"
Jul 29 01:19:45.149019 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.148981    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" containerName="manager"
Jul 29 01:19:45.149293 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.149065    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" containerName="manager"
Jul 29 01:19:45.149293 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.149077    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" containerName="console"
Jul 29 01:19:45.149293 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.149090    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" containerName="manager"
Jul 29 01:19:45.152590 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.152554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.173772 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.173748    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:19:45.187826 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.187784    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"77b9b2cb-2954-46a0-a972-7a288dc3119a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-gjphm\": pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.188248 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.188225    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:19:45.190359 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.190326    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.192196 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.192178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.235888 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.235854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:19:45.246987 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.246897    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.257931 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.257858    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.275498 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.275471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.275623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.275514    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5l9\" (UniqueName: \"kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.275623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.275546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.275623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.275576    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bjvr\" (UniqueName: \"kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.305726 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.305670    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.307304 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.307276    2564 generic.go:356] "Generic (PLEG): container finished" podID="e86d50ba-1036-4e00-b505-3693ae3157b5" containerID="b585bb908ba0074add486579818fbfa001c400cd57c78489ef9df52fe15e78b5" exitCode=0
Jul 29 01:19:45.311440 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.311375    2564 generic.go:356] "Generic (PLEG): container finished" podID="77b9b2cb-2954-46a0-a972-7a288dc3119a" containerID="bdf39cc5da7a94f1fc95e9c8247eead64f5f968f6dfb9daef0b9f6045b3425fb" exitCode=0
Jul 29 01:19:45.359680 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.359661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.364350 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.364325    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.366921 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.366901    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.376797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.376773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.376899 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.376813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5l9\" (UniqueName: \"kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.376899 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.376842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.377013 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.376889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bjvr\" (UniqueName: \"kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.377170 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.377132    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.377238 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.377215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.387610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.387586    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.391074 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.391047    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:45.392557 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.392514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5l9\" (UniqueName: \"kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9\") pod \"kuadrant-operator-controller-manager-86878c769f-kjsnw\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.393093 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.393076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bjvr\" (UniqueName: \"kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2lbcg\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.393433 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:45.393411    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:45.477770 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.477745    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2dvr\" (UniqueName: \"kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr\") pod \"e86d50ba-1036-4e00-b505-3693ae3157b5\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") "
Jul 29 01:19:45.477871 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.477806    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch7t9\" (UniqueName: \"kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9\") pod \"77b9b2cb-2954-46a0-a972-7a288dc3119a\" (UID: \"77b9b2cb-2954-46a0-a972-7a288dc3119a\") "
Jul 29 01:19:45.477871 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.477845    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume\") pod \"e86d50ba-1036-4e00-b505-3693ae3157b5\" (UID: \"e86d50ba-1036-4e00-b505-3693ae3157b5\") "
Jul 29 01:19:45.478364 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.478341    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e86d50ba-1036-4e00-b505-3693ae3157b5" (UID: "e86d50ba-1036-4e00-b505-3693ae3157b5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:45.479757 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.479736    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9" (OuterVolumeSpecName: "kube-api-access-ch7t9") pod "77b9b2cb-2954-46a0-a972-7a288dc3119a" (UID: "77b9b2cb-2954-46a0-a972-7a288dc3119a"). InnerVolumeSpecName "kube-api-access-ch7t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:45.479823 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.479764    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr" (OuterVolumeSpecName: "kube-api-access-h2dvr") pod "e86d50ba-1036-4e00-b505-3693ae3157b5" (UID: "e86d50ba-1036-4e00-b505-3693ae3157b5"). InnerVolumeSpecName "kube-api-access-h2dvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:45.501923 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.501902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:45.507915 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.507899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:45.579501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.579443    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2dvr\" (UniqueName: \"kubernetes.io/projected/e86d50ba-1036-4e00-b505-3693ae3157b5-kube-api-access-h2dvr\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:45.579501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.579471    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch7t9\" (UniqueName: \"kubernetes.io/projected/77b9b2cb-2954-46a0-a972-7a288dc3119a-kube-api-access-ch7t9\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:45.579501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.579482    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e86d50ba-1036-4e00-b505-3693ae3157b5-extensions-socket-volume\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:45.669902 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.669876    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:19:45.672709 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:45.672682    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd09b70e_cb04_4148_9dbd_1fe42e2aa625.slice/crio-17d3f5779d55fffb40f2b107b90de9b667f141d1776a2133287a2b38709ce6da WatchSource:0}: Error finding container 17d3f5779d55fffb40f2b107b90de9b667f141d1776a2133287a2b38709ce6da: Status 404 returned error can't find the container with id 17d3f5779d55fffb40f2b107b90de9b667f141d1776a2133287a2b38709ce6da
Jul 29 01:19:45.697364 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:19:45.697332    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113f68bb_5b88_47b5_9bf7_c37eb9543f8f.slice/crio-b73169f9654f50de5ff090e5e95f0e20a2425144802b4067ec97e1064e598e1c WatchSource:0}: Error finding container b73169f9654f50de5ff090e5e95f0e20a2425144802b4067ec97e1064e598e1c: Status 404 returned error can't find the container with id b73169f9654f50de5ff090e5e95f0e20a2425144802b4067ec97e1064e598e1c
Jul 29 01:19:45.699285 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:45.699261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:19:46.116645 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:46.116605    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gjphm\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:46.117657 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.117638    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b9b2cb-2954-46a0-a972-7a288dc3119a" path="/var/lib/kubelet/pods/77b9b2cb-2954-46a0-a972-7a288dc3119a/volumes"
Jul 29 01:19:46.118024 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.118011    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="842952c0-4de9-4ac3-9d58-bca1a4cfca51" path="/var/lib/kubelet/pods/842952c0-4de9-4ac3-9d58-bca1a4cfca51/volumes"
Jul 29 01:19:46.118359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.118346    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" path="/var/lib/kubelet/pods/e86d50ba-1036-4e00-b505-3693ae3157b5/volumes"
Jul 29 01:19:46.126684 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:46.126640    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2wsfk\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="e86d50ba-1036-4e00-b505-3693ae3157b5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:46.318062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.318030    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2wsfk"
Jul 29 01:19:46.318501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.318070    2564 scope.go:117] "RemoveContainer" containerID="b585bb908ba0074add486579818fbfa001c400cd57c78489ef9df52fe15e78b5"
Jul 29 01:19:46.319806 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.319775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" event={"ID":"113f68bb-5b88-47b5-9bf7-c37eb9543f8f","Type":"ContainerStarted","Data":"188c190156ea5d862ccd2fe7acab8ca8bf59a74f8a957a62eb549968743b7b88"}
Jul 29 01:19:46.319905 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.319841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:46.319905 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.319865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" event={"ID":"113f68bb-5b88-47b5-9bf7-c37eb9543f8f","Type":"ContainerStarted","Data":"b73169f9654f50de5ff090e5e95f0e20a2425144802b4067ec97e1064e598e1c"}
Jul 29 01:19:46.321615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.321593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" event={"ID":"bd09b70e-cb04-4148-9dbd-1fe42e2aa625","Type":"ContainerStarted","Data":"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"}
Jul 29 01:19:46.321718 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.321632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" event={"ID":"bd09b70e-cb04-4148-9dbd-1fe42e2aa625","Type":"ContainerStarted","Data":"17d3f5779d55fffb40f2b107b90de9b667f141d1776a2133287a2b38709ce6da"}
Jul 29 01:19:46.321718 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.321684    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:46.322948 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.322932    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gjphm"
Jul 29 01:19:46.328053 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.328037    2564 scope.go:117] "RemoveContainer" containerID="bdf39cc5da7a94f1fc95e9c8247eead64f5f968f6dfb9daef0b9f6045b3425fb"
Jul 29 01:19:46.383564 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.383516    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" podStartSLOduration=1.383501141 podStartE2EDuration="1.383501141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:46.353020171 +0000 UTC m=+440.784902651" watchObservedRunningTime="2026-07-29 01:19:46.383501141 +0000 UTC m=+440.815383620"
Jul 29 01:19:46.384672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:46.384643    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" podStartSLOduration=1.384635227 podStartE2EDuration="1.384635227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:19:46.382915244 +0000 UTC m=+440.814797724" watchObservedRunningTime="2026-07-29 01:19:46.384635227 +0000 UTC m=+440.816517707"
Jul 29 01:19:57.332232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.332138    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:19:57.332754 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.332236    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:57.428561 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.428531    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:19:57.428759 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.428719    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" podUID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" containerName="manager" containerID="cri-o://2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae" gracePeriod=10
Jul 29 01:19:57.674632 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.674607    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:57.779932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.779905    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bjvr\" (UniqueName: \"kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr\") pod \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") "
Jul 29 01:19:57.780068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.779952    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume\") pod \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\" (UID: \"bd09b70e-cb04-4148-9dbd-1fe42e2aa625\") "
Jul 29 01:19:57.780355 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.780329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bd09b70e-cb04-4148-9dbd-1fe42e2aa625" (UID: "bd09b70e-cb04-4148-9dbd-1fe42e2aa625"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:19:57.781768 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.781750    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr" (OuterVolumeSpecName: "kube-api-access-5bjvr") pod "bd09b70e-cb04-4148-9dbd-1fe42e2aa625" (UID: "bd09b70e-cb04-4148-9dbd-1fe42e2aa625"). InnerVolumeSpecName "kube-api-access-5bjvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:19:57.881362 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.881305    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bjvr\" (UniqueName: \"kubernetes.io/projected/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-kube-api-access-5bjvr\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:57.881362 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:57.881326    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd09b70e-cb04-4148-9dbd-1fe42e2aa625-extensions-socket-volume\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:19:58.393833 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.393797    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" containerID="2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae" exitCode=0
Jul 29 01:19:58.394236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.393844    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"
Jul 29 01:19:58.394236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.393853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" event={"ID":"bd09b70e-cb04-4148-9dbd-1fe42e2aa625","Type":"ContainerDied","Data":"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"}
Jul 29 01:19:58.394236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.393906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg" event={"ID":"bd09b70e-cb04-4148-9dbd-1fe42e2aa625","Type":"ContainerDied","Data":"17d3f5779d55fffb40f2b107b90de9b667f141d1776a2133287a2b38709ce6da"}
Jul 29 01:19:58.394236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.393935    2564 scope.go:117] "RemoveContainer" containerID="2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"
Jul 29 01:19:58.403015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.402997    2564 scope.go:117] "RemoveContainer" containerID="2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"
Jul 29 01:19:58.403306 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:19:58.403283    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae\": container with ID starting with 2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae not found: ID does not exist" containerID="2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"
Jul 29 01:19:58.403417 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.403312    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae"} err="failed to get container status \"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae\": rpc error: code = NotFound desc = could not find container \"2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae\": container with ID starting with 2a135c8c2ddbbf9730387cb30f295df40a0cb44b1f795a14700b463bce231eae not found: ID does not exist"
Jul 29 01:19:58.422556 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.422536    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:19:58.433906 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:19:58.433887    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2lbcg"]
Jul 29 01:20:00.116676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:00.116640    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" path="/var/lib/kubelet/pods/bd09b70e-cb04-4148-9dbd-1fe42e2aa625/volumes"
Jul 29 01:20:13.940551 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.940519    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"]
Jul 29 01:20:13.940981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.940914    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" containerName="manager"
Jul 29 01:20:13.940981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.940928    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" containerName="manager"
Jul 29 01:20:13.941092 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.941008    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd09b70e-cb04-4148-9dbd-1fe42e2aa625" containerName="manager"
Jul 29 01:20:13.943994 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.943980    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:13.947908 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.947892    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-f784k\""
Jul 29 01:20:13.969778 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:13.969750    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"]
Jul 29 01:20:14.096291 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096755 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.096755 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.096671    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s97gp\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-kube-api-access-s97gp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.197883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.197808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.197883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.197844    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s97gp\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-kube-api-access-s97gp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.197892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.197929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.197953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198467    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198599 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198639 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.198855 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.198836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.200541 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.200518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.200633 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.200558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.223551 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.223529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s97gp\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-kube-api-access-s97gp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.225377 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.225355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e13f08-b9f7-4782-84bd-0b62d4a33f76-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck9xh\" (UID: \"95e13f08-b9f7-4782-84bd-0b62d4a33f76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.253608 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.253585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:14.454242 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:14.454198    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e13f08_b9f7_4782_84bd_0b62d4a33f76.slice/crio-27e7890bf0a886a07220725937ec3e67f7aa3321efe76a218af0fed4239dfacb WatchSource:0}: Error finding container 27e7890bf0a886a07220725937ec3e67f7aa3321efe76a218af0fed4239dfacb: Status 404 returned error can't find the container with id 27e7890bf0a886a07220725937ec3e67f7aa3321efe76a218af0fed4239dfacb
Jul 29 01:20:14.456316 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.456288    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:20:14.456387 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.456352    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:20:14.456387 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.456379    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:20:14.480076 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.480048    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"]
Jul 29 01:20:14.480603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:14.480583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" event={"ID":"95e13f08-b9f7-4782-84bd-0b62d4a33f76","Type":"ContainerStarted","Data":"27e7890bf0a886a07220725937ec3e67f7aa3321efe76a218af0fed4239dfacb"}
Jul 29 01:20:15.487923 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:15.487890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" event={"ID":"95e13f08-b9f7-4782-84bd-0b62d4a33f76","Type":"ContainerStarted","Data":"98f52b4d0fa79b5cc7dc9632a921dd218f3a882d439258561b59f337e047cb56"}
Jul 29 01:20:16.254549 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.254511    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:16.259331 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.259306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:16.305418 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.305356    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podStartSLOduration=3.305341178 podStartE2EDuration="3.305341178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:15.527888055 +0000 UTC m=+469.959770535" watchObservedRunningTime="2026-07-29 01:20:16.305341178 +0000 UTC m=+470.737223658"
Jul 29 01:20:16.493715 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.493682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:16.494504 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.494485    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:20:16.689969 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.689926    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:16.693130 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.693109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.696347 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.696327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t5xr4\""
Jul 29 01:20:16.696459 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.696327    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 01:20:16.709562 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.709536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:16.785927 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.785886    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:16.818750 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.818714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf4tw\" (UniqueName: \"kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.818890 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.818754    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.920259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.920225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf4tw\" (UniqueName: \"kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.920416 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.920267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.921018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.920992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:16.932449 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:16.932426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf4tw\" (UniqueName: \"kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw\") pod \"limitador-limitador-64c94fd64f-sw2jc\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:17.004616 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.004555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:17.174755 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.174725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:17.176905 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:17.176880    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae52c76_6786_46d0_ad5a_9952064aae62.slice/crio-79f03a16f1395e03cf12e8e17bd4e2462c062d9256822571a7ff272e8da70a9d WatchSource:0}: Error finding container 79f03a16f1395e03cf12e8e17bd4e2462c062d9256822571a7ff272e8da70a9d: Status 404 returned error can't find the container with id 79f03a16f1395e03cf12e8e17bd4e2462c062d9256822571a7ff272e8da70a9d
Jul 29 01:20:17.500124 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.500091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" event={"ID":"6ae52c76-6786-46d0-ad5a-9952064aae62","Type":"ContainerStarted","Data":"79f03a16f1395e03cf12e8e17bd4e2462c062d9256822571a7ff272e8da70a9d"}
Jul 29 01:20:17.644456 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.644427    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:17.647556 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.647523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:17.654416 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.654395    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hrm72\""
Jul 29 01:20:17.664697 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.664675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:17.727421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.727395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvpd\" (UniqueName: \"kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd\") pod \"authorino-58cd955675-klqhw\" (UID: \"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6\") " pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:17.828248 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.828220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvpd\" (UniqueName: \"kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd\") pod \"authorino-58cd955675-klqhw\" (UID: \"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6\") " pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:17.852950 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.852916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvpd\" (UniqueName: \"kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd\") pod \"authorino-58cd955675-klqhw\" (UID: \"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6\") " pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:17.857663 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.857640    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:17.860322 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.860303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:17.876429 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.876406    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:17.929634 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.929603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9ls\" (UniqueName: \"kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls\") pod \"authorino-75d556bd77-mxfr8\" (UID: \"bcf1f617-d337-445b-99ac-8ccd6bf224b7\") " pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:17.957312 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:17.957286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:18.031139 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.031102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9ls\" (UniqueName: \"kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls\") pod \"authorino-75d556bd77-mxfr8\" (UID: \"bcf1f617-d337-445b-99ac-8ccd6bf224b7\") " pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:18.051022 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.050985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9ls\" (UniqueName: \"kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls\") pod \"authorino-75d556bd77-mxfr8\" (UID: \"bcf1f617-d337-445b-99ac-8ccd6bf224b7\") " pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:18.164132 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.164081    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:18.170374 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:18.170344    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod483dc4d6_d5a1_45e6_b27a_8be4f2d8d7b6.slice/crio-101f50693fa50c3c648e614046f7c18efb1bd5f70dae2b5910779f6f25126eca WatchSource:0}: Error finding container 101f50693fa50c3c648e614046f7c18efb1bd5f70dae2b5910779f6f25126eca: Status 404 returned error can't find the container with id 101f50693fa50c3c648e614046f7c18efb1bd5f70dae2b5910779f6f25126eca
Jul 29 01:20:18.178302 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.178070    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:18.360315 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.360283    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:18.361956 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:18.361925    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcf1f617_d337_445b_99ac_8ccd6bf224b7.slice/crio-b64b5559142ae504abde77488fcdedb8b1ad69e4cf61e4ddabd35b515039d73e WatchSource:0}: Error finding container b64b5559142ae504abde77488fcdedb8b1ad69e4cf61e4ddabd35b515039d73e: Status 404 returned error can't find the container with id b64b5559142ae504abde77488fcdedb8b1ad69e4cf61e4ddabd35b515039d73e
Jul 29 01:20:18.508133 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.508099    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mxfr8" event={"ID":"bcf1f617-d337-445b-99ac-8ccd6bf224b7","Type":"ContainerStarted","Data":"b64b5559142ae504abde77488fcdedb8b1ad69e4cf61e4ddabd35b515039d73e"}
Jul 29 01:20:18.509527 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:18.509501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-klqhw" event={"ID":"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6","Type":"ContainerStarted","Data":"101f50693fa50c3c648e614046f7c18efb1bd5f70dae2b5910779f6f25126eca"}
Jul 29 01:20:20.527569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:20.527509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" event={"ID":"6ae52c76-6786-46d0-ad5a-9952064aae62","Type":"ContainerStarted","Data":"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"}
Jul 29 01:20:20.528011 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:20.527883    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:20.552797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:20.552729    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" podStartSLOduration=1.42437253 podStartE2EDuration="4.552711226s" totalImagesPullingTime="3.128338696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:20.551363069 +0000 UTC m=+474.983245550" watchObservedRunningTime="2026-07-29 01:20:20.552711226 +0000 UTC m=+474.984593707"
Jul 29 01:20:22.541461 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:22.541423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mxfr8" event={"ID":"bcf1f617-d337-445b-99ac-8ccd6bf224b7","Type":"ContainerStarted","Data":"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"}
Jul 29 01:20:22.543318 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:22.543279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-klqhw" event={"ID":"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6","Type":"ContainerStarted","Data":"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"}
Jul 29 01:20:22.570615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:22.570570    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mxfr8" podStartSLOduration=1.582671033 podStartE2EDuration="5.570557834s" totalImagesPullingTime="3.987886801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:22.568725495 +0000 UTC m=+477.000607985" watchObservedRunningTime="2026-07-29 01:20:22.570557834 +0000 UTC m=+477.002440314"
Jul 29 01:20:22.646460 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:22.646354    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-klqhw" podStartSLOduration=1.458427732 podStartE2EDuration="5.646337611s" totalImagesPullingTime="4.187909879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:22.605538313 +0000 UTC m=+477.037420828" watchObservedRunningTime="2026-07-29 01:20:22.646337611 +0000 UTC m=+477.078220090"
Jul 29 01:20:22.648320 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:22.648290    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:24.561928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:24.561883    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-klqhw" podUID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" containerName="authorino" containerID="cri-o://6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf" gracePeriod=30
Jul 29 01:20:24.804859 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:24.804834    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:24.899990 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:24.899928    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpvpd\" (UniqueName: \"kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd\") pod \"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6\" (UID: \"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6\") "
Jul 29 01:20:24.901920 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:24.901898    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd" (OuterVolumeSpecName: "kube-api-access-wpvpd") pod "483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" (UID: "483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6"). InnerVolumeSpecName "kube-api-access-wpvpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:20:25.000652 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.000625    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpvpd\" (UniqueName: \"kubernetes.io/projected/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6-kube-api-access-wpvpd\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:25.569436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.569402    2564 generic.go:356] "Generic (PLEG): container finished" podID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" containerID="6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf" exitCode=0
Jul 29 01:20:25.569436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.569437    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-klqhw"
Jul 29 01:20:25.569895 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.569467    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-klqhw" event={"ID":"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6","Type":"ContainerDied","Data":"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"}
Jul 29 01:20:25.569895 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.569501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-klqhw" event={"ID":"483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6","Type":"ContainerDied","Data":"101f50693fa50c3c648e614046f7c18efb1bd5f70dae2b5910779f6f25126eca"}
Jul 29 01:20:25.569895 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.569519    2564 scope.go:117] "RemoveContainer" containerID="6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"
Jul 29 01:20:25.580233 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.580212    2564 scope.go:117] "RemoveContainer" containerID="6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"
Jul 29 01:20:25.580532 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:20:25.580511    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf\": container with ID starting with 6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf not found: ID does not exist" containerID="6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"
Jul 29 01:20:25.580601 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.580540    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf"} err="failed to get container status \"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf\": rpc error: code = NotFound desc = could not find container \"6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf\": container with ID starting with 6bf774ca7e0a31d0d04acfa1dbe062cd84697c0810364265b0a3b11abb56d1cf not found: ID does not exist"
Jul 29 01:20:25.594245 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.594215    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:25.601223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:25.601204    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-klqhw"]
Jul 29 01:20:26.116705 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:26.116678    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" path="/var/lib/kubelet/pods/483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6/volumes"
Jul 29 01:20:31.534781 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:31.534753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:31.819015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:31.818930    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:31.819186 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:31.819131    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" podUID="6ae52c76-6786-46d0-ad5a-9952064aae62" containerName="limitador" containerID="cri-o://e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076" gracePeriod=30
Jul 29 01:20:32.375974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.375948    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:32.555388 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.555361    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf4tw\" (UniqueName: \"kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw\") pod \"6ae52c76-6786-46d0-ad5a-9952064aae62\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") "
Jul 29 01:20:32.555756 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.555455    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file\") pod \"6ae52c76-6786-46d0-ad5a-9952064aae62\" (UID: \"6ae52c76-6786-46d0-ad5a-9952064aae62\") "
Jul 29 01:20:32.555822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.555801    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file" (OuterVolumeSpecName: "config-file") pod "6ae52c76-6786-46d0-ad5a-9952064aae62" (UID: "6ae52c76-6786-46d0-ad5a-9952064aae62"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:20:32.557559 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.557528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw" (OuterVolumeSpecName: "kube-api-access-pf4tw") pod "6ae52c76-6786-46d0-ad5a-9952064aae62" (UID: "6ae52c76-6786-46d0-ad5a-9952064aae62"). InnerVolumeSpecName "kube-api-access-pf4tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:20:32.613281 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.613257    2564 generic.go:356] "Generic (PLEG): container finished" podID="6ae52c76-6786-46d0-ad5a-9952064aae62" containerID="e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076" exitCode=0
Jul 29 01:20:32.613386 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.613303    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"
Jul 29 01:20:32.613386 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.613345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" event={"ID":"6ae52c76-6786-46d0-ad5a-9952064aae62","Type":"ContainerDied","Data":"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"}
Jul 29 01:20:32.613465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.613394    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-sw2jc" event={"ID":"6ae52c76-6786-46d0-ad5a-9952064aae62","Type":"ContainerDied","Data":"79f03a16f1395e03cf12e8e17bd4e2462c062d9256822571a7ff272e8da70a9d"}
Jul 29 01:20:32.613465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.613416    2564 scope.go:117] "RemoveContainer" containerID="e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"
Jul 29 01:20:32.622277 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.622242    2564 scope.go:117] "RemoveContainer" containerID="e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"
Jul 29 01:20:32.622513 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:20:32.622496    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076\": container with ID starting with e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076 not found: ID does not exist" containerID="e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"
Jul 29 01:20:32.622576 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.622526    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076"} err="failed to get container status \"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076\": rpc error: code = NotFound desc = could not find container \"e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076\": container with ID starting with e06e49dc768262ed04bfd18f4aca7aa8045faf583f15312d769f7897655e8076 not found: ID does not exist"
Jul 29 01:20:32.647186 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.647159    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:32.656077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.656058    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6ae52c76-6786-46d0-ad5a-9952064aae62-config-file\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:32.656166 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.656087    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf4tw\" (UniqueName: \"kubernetes.io/projected/6ae52c76-6786-46d0-ad5a-9952064aae62-kube-api-access-pf4tw\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:32.660791 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:32.660770    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-sw2jc"]
Jul 29 01:20:34.117285 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:34.117253    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae52c76-6786-46d0-ad5a-9952064aae62" path="/var/lib/kubelet/pods/6ae52c76-6786-46d0-ad5a-9952064aae62/volumes"
Jul 29 01:20:38.286466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286426    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"]
Jul 29 01:20:38.286820 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286782    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae52c76-6786-46d0-ad5a-9952064aae62" containerName="limitador"
Jul 29 01:20:38.286820 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286793    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae52c76-6786-46d0-ad5a-9952064aae62" containerName="limitador"
Jul 29 01:20:38.286820 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286817    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" containerName="authorino"
Jul 29 01:20:38.286926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" containerName="authorino"
Jul 29 01:20:38.286926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286898    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="483dc4d6-d5a1-45e6-b27a-8be4f2d8d7b6" containerName="authorino"
Jul 29 01:20:38.286926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.286906    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae52c76-6786-46d0-ad5a-9952064aae62" containerName="limitador"
Jul 29 01:20:38.291055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.291032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.303541 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.303516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 01:20:38.303541 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.303541    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 01:20:38.304522 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.304504    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:20:38.304625 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.304505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gcjhs\""
Jul 29 01:20:38.321580 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.321558    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"]
Jul 29 01:20:38.403938 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.403910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcpk7\" (UniqueName: \"kubernetes.io/projected/f2e0be6e-d16d-4fe8-9b16-da7099767911-kube-api-access-xcpk7\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.404076 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.403965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f2e0be6e-d16d-4fe8-9b16-da7099767911-data\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.504889 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.504862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f2e0be6e-d16d-4fe8-9b16-da7099767911-data\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.505017 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.504924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpk7\" (UniqueName: \"kubernetes.io/projected/f2e0be6e-d16d-4fe8-9b16-da7099767911-kube-api-access-xcpk7\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.505236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.505218    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f2e0be6e-d16d-4fe8-9b16-da7099767911-data\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.534002 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.533974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpk7\" (UniqueName: \"kubernetes.io/projected/f2e0be6e-d16d-4fe8-9b16-da7099767911-kube-api-access-xcpk7\") pod \"postgres-69fb4594cf-qswtb\" (UID: \"f2e0be6e-d16d-4fe8-9b16-da7099767911\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.600534 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.600470    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:38.738739 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:38.738711    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"]
Jul 29 01:20:38.739588 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:38.739557    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e0be6e_d16d_4fe8_9b16_da7099767911.slice/crio-7e9f5c9b3b4577904f73c7b13e2fbc980781dea5d1fce295907387bba782563f WatchSource:0}: Error finding container 7e9f5c9b3b4577904f73c7b13e2fbc980781dea5d1fce295907387bba782563f: Status 404 returned error can't find the container with id 7e9f5c9b3b4577904f73c7b13e2fbc980781dea5d1fce295907387bba782563f
Jul 29 01:20:39.656521 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:39.656472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb" event={"ID":"f2e0be6e-d16d-4fe8-9b16-da7099767911","Type":"ContainerStarted","Data":"7e9f5c9b3b4577904f73c7b13e2fbc980781dea5d1fce295907387bba782563f"}
Jul 29 01:20:44.688968 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:44.688929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb" event={"ID":"f2e0be6e-d16d-4fe8-9b16-da7099767911","Type":"ContainerStarted","Data":"e498d58aed2ea70afa9561cf07d30e93901872ddc8ac36f33ed9d480855394d3"}
Jul 29 01:20:44.689444 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:44.689009    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:44.722640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:44.722593    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb" podStartSLOduration=1.757845627 podStartE2EDuration="6.722576931s" totalImagesPullingTime="4.964731304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:44.721267705 +0000 UTC m=+499.153150187" watchObservedRunningTime="2026-07-29 01:20:44.722576931 +0000 UTC m=+499.154459417"
Jul 29 01:20:50.723391 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:50.723366    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qswtb"
Jul 29 01:20:51.319186 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.319128    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:51.322672 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.322649    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:51.330625 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.330602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:51.409955 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.409922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqc44\" (UniqueName: \"kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44\") pod \"authorino-5d4b67c446-fbszf\" (UID: \"95f952df-59e4-4fa0-bba9-6812d809c9c3\") " pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:51.511406 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.511373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc44\" (UniqueName: \"kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44\") pod \"authorino-5d4b67c446-fbszf\" (UID: \"95f952df-59e4-4fa0-bba9-6812d809c9c3\") " pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:51.523819 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.523798    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqc44\" (UniqueName: \"kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44\") pod \"authorino-5d4b67c446-fbszf\" (UID: \"95f952df-59e4-4fa0-bba9-6812d809c9c3\") " pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:51.541684 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.541657    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:51.541892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.541880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:51.563627 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.563603    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:51.567201 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.567183    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:51.575020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.574731    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:51.613018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.612942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kksw\" (UniqueName: \"kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw\") pod \"authorino-56656c8796-8ptpn\" (UID: \"6beb2609-5955-4fae-b7c2-c869586488bb\") " pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:51.682758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.682735    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:51.684533 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:51.684508    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95f952df_59e4_4fa0_bba9_6812d809c9c3.slice/crio-89eab416f2dd9e80b1ff10579a07b0d32f80187af5f7e43924501c2dc590d5de WatchSource:0}: Error finding container 89eab416f2dd9e80b1ff10579a07b0d32f80187af5f7e43924501c2dc590d5de: Status 404 returned error can't find the container with id 89eab416f2dd9e80b1ff10579a07b0d32f80187af5f7e43924501c2dc590d5de
Jul 29 01:20:51.714255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.714233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kksw\" (UniqueName: \"kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw\") pod \"authorino-56656c8796-8ptpn\" (UID: \"6beb2609-5955-4fae-b7c2-c869586488bb\") " pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:51.722867 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.722849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kksw\" (UniqueName: \"kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw\") pod \"authorino-56656c8796-8ptpn\" (UID: \"6beb2609-5955-4fae-b7c2-c869586488bb\") " pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:51.727911 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.727881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fbszf" event={"ID":"95f952df-59e4-4fa0-bba9-6812d809c9c3","Type":"ContainerStarted","Data":"89eab416f2dd9e80b1ff10579a07b0d32f80187af5f7e43924501c2dc590d5de"}
Jul 29 01:20:51.847981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.847904    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:51.848172 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.848160    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:51.877861 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.877832    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:20:51.881624 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.881603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:51.884674 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.884652    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 01:20:51.891513 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.891490    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:20:51.916432 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.916404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:51.916547 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.916465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf889\" (UniqueName: \"kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:51.986189 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:51.986162    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:51.987254 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:51.987224    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6beb2609_5955_4fae_b7c2_c869586488bb.slice/crio-f7e551c06c91168c38b0ad36e20349e99376d77330b0c4791aacd7145fd764ac WatchSource:0}: Error finding container f7e551c06c91168c38b0ad36e20349e99376d77330b0c4791aacd7145fd764ac: Status 404 returned error can't find the container with id f7e551c06c91168c38b0ad36e20349e99376d77330b0c4791aacd7145fd764ac
Jul 29 01:20:52.017826 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.017802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:52.017892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.017868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf889\" (UniqueName: \"kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:52.020302 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.020276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:52.027239 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.027224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf889\" (UniqueName: \"kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889\") pod \"authorino-6d88648b44-qcl7m\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") " pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:52.192899 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.192805    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:20:52.334352 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.334325    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:20:52.335008 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:52.334987    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod631b4472_2d90_44cd_8ce5_793d7fabbea1.slice/crio-6aa3b550464137de615bfd95b41faac196d245c5e7e928eb30fd7d6198875520 WatchSource:0}: Error finding container 6aa3b550464137de615bfd95b41faac196d245c5e7e928eb30fd7d6198875520: Status 404 returned error can't find the container with id 6aa3b550464137de615bfd95b41faac196d245c5e7e928eb30fd7d6198875520
Jul 29 01:20:52.736765 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.736732    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d88648b44-qcl7m" event={"ID":"631b4472-2d90-44cd-8ce5-793d7fabbea1","Type":"ContainerStarted","Data":"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"}
Jul 29 01:20:52.737125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.736774    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d88648b44-qcl7m" event={"ID":"631b4472-2d90-44cd-8ce5-793d7fabbea1","Type":"ContainerStarted","Data":"6aa3b550464137de615bfd95b41faac196d245c5e7e928eb30fd7d6198875520"}
Jul 29 01:20:52.738505 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.738482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56656c8796-8ptpn" event={"ID":"6beb2609-5955-4fae-b7c2-c869586488bb","Type":"ContainerStarted","Data":"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"}
Jul 29 01:20:52.738615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.738518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56656c8796-8ptpn" event={"ID":"6beb2609-5955-4fae-b7c2-c869586488bb","Type":"ContainerStarted","Data":"f7e551c06c91168c38b0ad36e20349e99376d77330b0c4791aacd7145fd764ac"}
Jul 29 01:20:52.738615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.738479    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56656c8796-8ptpn" podUID="6beb2609-5955-4fae-b7c2-c869586488bb" containerName="authorino" containerID="cri-o://f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20" gracePeriod=30
Jul 29 01:20:52.740134 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.740076    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fbszf" event={"ID":"95f952df-59e4-4fa0-bba9-6812d809c9c3","Type":"ContainerStarted","Data":"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"}
Jul 29 01:20:52.740273 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.740136    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fbszf" podUID="95f952df-59e4-4fa0-bba9-6812d809c9c3" containerName="authorino" containerID="cri-o://63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b" gracePeriod=30
Jul 29 01:20:52.757328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.757288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d88648b44-qcl7m" podStartSLOduration=1.449778585 podStartE2EDuration="1.757275424s" totalImagesPullingTime="307.496839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:52.754707275 +0000 UTC m=+507.186589767" watchObservedRunningTime="2026-07-29 01:20:52.757275424 +0000 UTC m=+507.189157903"
Jul 29 01:20:52.782600 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.782504    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56656c8796-8ptpn" podStartSLOduration=1.256784018 podStartE2EDuration="1.782463391s" totalImagesPullingTime="525.679373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:52.779518975 +0000 UTC m=+507.211401456" watchObservedRunningTime="2026-07-29 01:20:52.782463391 +0000 UTC m=+507.214345868"
Jul 29 01:20:52.804725 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.804694    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:52.804944 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.804923    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mxfr8" podUID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" containerName="authorino" containerID="cri-o://2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c" gracePeriod=30
Jul 29 01:20:52.815177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:52.815118    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fbszf" podStartSLOduration=1.4888553660000001 podStartE2EDuration="1.815103463s" totalImagesPullingTime="326.248097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:52.811989845 +0000 UTC m=+507.243872328" watchObservedRunningTime="2026-07-29 01:20:52.815103463 +0000 UTC m=+507.246985943"
Jul 29 01:20:53.080839 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.080810    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:53.126465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.126435    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqc44\" (UniqueName: \"kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44\") pod \"95f952df-59e4-4fa0-bba9-6812d809c9c3\" (UID: \"95f952df-59e4-4fa0-bba9-6812d809c9c3\") "
Jul 29 01:20:53.128515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.128486    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44" (OuterVolumeSpecName: "kube-api-access-hqc44") pod "95f952df-59e4-4fa0-bba9-6812d809c9c3" (UID: "95f952df-59e4-4fa0-bba9-6812d809c9c3"). InnerVolumeSpecName "kube-api-access-hqc44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:20:53.131724 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.131696    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:53.134336 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.134316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:53.227336 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.227313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kksw\" (UniqueName: \"kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw\") pod \"6beb2609-5955-4fae-b7c2-c869586488bb\" (UID: \"6beb2609-5955-4fae-b7c2-c869586488bb\") "
Jul 29 01:20:53.227442 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.227402    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz9ls\" (UniqueName: \"kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls\") pod \"bcf1f617-d337-445b-99ac-8ccd6bf224b7\" (UID: \"bcf1f617-d337-445b-99ac-8ccd6bf224b7\") "
Jul 29 01:20:53.227662 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.227645    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqc44\" (UniqueName: \"kubernetes.io/projected/95f952df-59e4-4fa0-bba9-6812d809c9c3-kube-api-access-hqc44\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:53.229495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.229472    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls" (OuterVolumeSpecName: "kube-api-access-kz9ls") pod "bcf1f617-d337-445b-99ac-8ccd6bf224b7" (UID: "bcf1f617-d337-445b-99ac-8ccd6bf224b7"). InnerVolumeSpecName "kube-api-access-kz9ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:20:53.229572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.229551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw" (OuterVolumeSpecName: "kube-api-access-8kksw") pod "6beb2609-5955-4fae-b7c2-c869586488bb" (UID: "6beb2609-5955-4fae-b7c2-c869586488bb"). InnerVolumeSpecName "kube-api-access-8kksw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:20:53.328179 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.328105    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz9ls\" (UniqueName: \"kubernetes.io/projected/bcf1f617-d337-445b-99ac-8ccd6bf224b7-kube-api-access-kz9ls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:53.328179 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.328130    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kksw\" (UniqueName: \"kubernetes.io/projected/6beb2609-5955-4fae-b7c2-c869586488bb-kube-api-access-8kksw\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:20:53.747022 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.746945    2564 generic.go:356] "Generic (PLEG): container finished" podID="6beb2609-5955-4fae-b7c2-c869586488bb" containerID="f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20" exitCode=0
Jul 29 01:20:53.747022 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.746990    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56656c8796-8ptpn"
Jul 29 01:20:53.747502 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.747037    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56656c8796-8ptpn" event={"ID":"6beb2609-5955-4fae-b7c2-c869586488bb","Type":"ContainerDied","Data":"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"}
Jul 29 01:20:53.747502 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.747083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56656c8796-8ptpn" event={"ID":"6beb2609-5955-4fae-b7c2-c869586488bb","Type":"ContainerDied","Data":"f7e551c06c91168c38b0ad36e20349e99376d77330b0c4791aacd7145fd764ac"}
Jul 29 01:20:53.747502 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.747104    2564 scope.go:117] "RemoveContainer" containerID="f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"
Jul 29 01:20:53.748544 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.748526    2564 generic.go:356] "Generic (PLEG): container finished" podID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" containerID="2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c" exitCode=0
Jul 29 01:20:53.748603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.748576    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mxfr8"
Jul 29 01:20:53.748645 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.748615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mxfr8" event={"ID":"bcf1f617-d337-445b-99ac-8ccd6bf224b7","Type":"ContainerDied","Data":"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"}
Jul 29 01:20:53.748708 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.748654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mxfr8" event={"ID":"bcf1f617-d337-445b-99ac-8ccd6bf224b7","Type":"ContainerDied","Data":"b64b5559142ae504abde77488fcdedb8b1ad69e4cf61e4ddabd35b515039d73e"}
Jul 29 01:20:53.749892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.749870    2564 generic.go:356] "Generic (PLEG): container finished" podID="95f952df-59e4-4fa0-bba9-6812d809c9c3" containerID="63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b" exitCode=0
Jul 29 01:20:53.750015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.749915    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fbszf"
Jul 29 01:20:53.750015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.749949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fbszf" event={"ID":"95f952df-59e4-4fa0-bba9-6812d809c9c3","Type":"ContainerDied","Data":"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"}
Jul 29 01:20:53.750015 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.749974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fbszf" event={"ID":"95f952df-59e4-4fa0-bba9-6812d809c9c3","Type":"ContainerDied","Data":"89eab416f2dd9e80b1ff10579a07b0d32f80187af5f7e43924501c2dc590d5de"}
Jul 29 01:20:53.758393 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.758370    2564 scope.go:117] "RemoveContainer" containerID="f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"
Jul 29 01:20:53.758673 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:20:53.758653    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20\": container with ID starting with f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20 not found: ID does not exist" containerID="f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"
Jul 29 01:20:53.758742 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.758678    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20"} err="failed to get container status \"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20\": rpc error: code = NotFound desc = could not find container \"f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20\": container with ID starting with f415526e2f7d11a5a1ddfe760bc393dde1bc0cf8e3cff656e7aa57978bd9be20 not found: ID does not exist"
Jul 29 01:20:53.758742 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.758694    2564 scope.go:117] "RemoveContainer" containerID="2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"
Jul 29 01:20:53.768169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.768139    2564 scope.go:117] "RemoveContainer" containerID="2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"
Jul 29 01:20:53.770605 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:20:53.769635    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c\": container with ID starting with 2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c not found: ID does not exist" containerID="2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"
Jul 29 01:20:53.770605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.769671    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c"} err="failed to get container status \"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c\": rpc error: code = NotFound desc = could not find container \"2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c\": container with ID starting with 2618877005d8bd506e6376368a1298fd517074f75fd7908babf946bdd800c11c not found: ID does not exist"
Jul 29 01:20:53.770605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.769695    2564 scope.go:117] "RemoveContainer" containerID="63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"
Jul 29 01:20:53.779021 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.779000    2564 scope.go:117] "RemoveContainer" containerID="63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"
Jul 29 01:20:53.779318 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:20:53.779286    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b\": container with ID starting with 63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b not found: ID does not exist" containerID="63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"
Jul 29 01:20:53.779390 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.779325    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b"} err="failed to get container status \"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b\": rpc error: code = NotFound desc = could not find container \"63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b\": container with ID starting with 63a640acce492cb2b9e986631d6faa13afcdf001c3a03367ccb7f81696230f2b not found: ID does not exist"
Jul 29 01:20:53.782358 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.782337    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:53.792183 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.792160    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56656c8796-8ptpn"]
Jul 29 01:20:53.843931 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.843912    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:53.850001 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.849981    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fbszf"]
Jul 29 01:20:53.868430 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.868410    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:53.874289 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:53.874271    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mxfr8"]
Jul 29 01:20:54.117645 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:54.117612    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6beb2609-5955-4fae-b7c2-c869586488bb" path="/var/lib/kubelet/pods/6beb2609-5955-4fae-b7c2-c869586488bb/volumes"
Jul 29 01:20:54.118025 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:54.118006    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95f952df-59e4-4fa0-bba9-6812d809c9c3" path="/var/lib/kubelet/pods/95f952df-59e4-4fa0-bba9-6812d809c9c3/volumes"
Jul 29 01:20:54.118381 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:54.118363    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" path="/var/lib/kubelet/pods/bcf1f617-d337-445b-99ac-8ccd6bf224b7/volumes"
Jul 29 01:20:57.924036 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924006    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:20:57.924466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924429    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" containerName="authorino"
Jul 29 01:20:57.924466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924443    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" containerName="authorino"
Jul 29 01:20:57.924466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924466    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95f952df-59e4-4fa0-bba9-6812d809c9c3" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924471    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="95f952df-59e4-4fa0-bba9-6812d809c9c3" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924479    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6beb2609-5955-4fae-b7c2-c869586488bb" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924485    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6beb2609-5955-4fae-b7c2-c869586488bb" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924550    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf1f617-d337-445b-99ac-8ccd6bf224b7" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924558    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6beb2609-5955-4fae-b7c2-c869586488bb" containerName="authorino"
Jul 29 01:20:57.924573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:57.924563    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="95f952df-59e4-4fa0-bba9-6812d809c9c3" containerName="authorino"
Jul 29 01:20:58.093189 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.093135    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:20:58.093357 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.093260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.096812 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.096792    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 01:20:58.096937 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.096791    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 01:20:58.098085 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.098068    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z4mq5\""
Jul 29 01:20:58.149236 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.149211    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:20:58.166304 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.166279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.166424 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.166343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfdgs\" (UniqueName: \"kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.173571 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.173554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:20:58.173665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.173635    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.266852 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.266824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gbgb\" (UniqueName: \"kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.266981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.266894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.266981 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.266960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfdgs\" (UniqueName: \"kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.267095 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.267003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.269216 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.269193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.277249 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.277231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfdgs\" (UniqueName: \"kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs\") pod \"maas-controller-54666f4746-wps7x\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") " pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.368321 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.368296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.368424 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.368332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gbgb\" (UniqueName: \"kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.370426 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.370407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.378277 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.378258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gbgb\" (UniqueName: \"kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb\") pod \"maas-controller-6dd887db67-pzlbp\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") " pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.404242 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.404214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:20:58.483295 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.483266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:20:58.548012 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.547916    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:20:58.627395 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.627372    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:20:58.628339 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:20:58.628308    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401d870b_0699_4a36_9dc7_8e5ffdfe5e03.slice/crio-3c60bf6a481013be09d9a6e3751965c53fe021e785c5e366e5b716226922e9ac WatchSource:0}: Error finding container 3c60bf6a481013be09d9a6e3751965c53fe021e785c5e366e5b716226922e9ac: Status 404 returned error can't find the container with id 3c60bf6a481013be09d9a6e3751965c53fe021e785c5e366e5b716226922e9ac
Jul 29 01:20:58.783223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.783189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dd887db67-pzlbp" event={"ID":"401d870b-0699-4a36-9dc7-8e5ffdfe5e03","Type":"ContainerStarted","Data":"3c60bf6a481013be09d9a6e3751965c53fe021e785c5e366e5b716226922e9ac"}
Jul 29 01:20:58.784352 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:20:58.784332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54666f4746-wps7x" event={"ID":"614c1594-2930-4926-b391-6c4496bd0bec","Type":"ContainerStarted","Data":"054d3a4ba0cd76884859dc50874bb07dac03ebba784e2adba84bb97f25066536"}
Jul 29 01:21:01.808914 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.808873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54666f4746-wps7x" event={"ID":"614c1594-2930-4926-b391-6c4496bd0bec","Type":"ContainerStarted","Data":"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"}
Jul 29 01:21:01.809360 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.808948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:21:01.810433 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.810408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dd887db67-pzlbp" event={"ID":"401d870b-0699-4a36-9dc7-8e5ffdfe5e03","Type":"ContainerStarted","Data":"d67169d0ca9fa9e89a3aa812c036b7f6d2a33f3819f7e7e9009b89e8820f8a9e"}
Jul 29 01:21:01.810567 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.810547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:21:01.837695 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.837645    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54666f4746-wps7x" podStartSLOduration=2.355805541 podStartE2EDuration="4.837631641s" totalImagesPullingTime="2.4818261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:21:01.835116525 +0000 UTC m=+516.266999004" watchObservedRunningTime="2026-07-29 01:21:01.837631641 +0000 UTC m=+516.269514121"
Jul 29 01:21:01.859243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:01.859204    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dd887db67-pzlbp" podStartSLOduration=1.472600072 podStartE2EDuration="3.859192193s" totalImagesPullingTime="2.386592121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:21:01.857440753 +0000 UTC m=+516.289323258" watchObservedRunningTime="2026-07-29 01:21:01.859192193 +0000 UTC m=+516.291074676"
Jul 29 01:21:12.819422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:12.819389    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:21:12.819822 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:12.819564    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:21:12.909096 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:12.909063    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:21:12.909323 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:12.909285    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54666f4746-wps7x" podUID="614c1594-2930-4926-b391-6c4496bd0bec" containerName="manager" containerID="cri-o://4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa" gracePeriod=30
Jul 29 01:21:13.162068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.162044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:21:13.296374 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.296348    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfdgs\" (UniqueName: \"kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs\") pod \"614c1594-2930-4926-b391-6c4496bd0bec\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") "
Jul 29 01:21:13.296496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.296451    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert\") pod \"614c1594-2930-4926-b391-6c4496bd0bec\" (UID: \"614c1594-2930-4926-b391-6c4496bd0bec\") "
Jul 29 01:21:13.298493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.298466    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "614c1594-2930-4926-b391-6c4496bd0bec" (UID: "614c1594-2930-4926-b391-6c4496bd0bec"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:21:13.298601 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.298505    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs" (OuterVolumeSpecName: "kube-api-access-gfdgs") pod "614c1594-2930-4926-b391-6c4496bd0bec" (UID: "614c1594-2930-4926-b391-6c4496bd0bec"). InnerVolumeSpecName "kube-api-access-gfdgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:21:13.397504 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.397438    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/614c1594-2930-4926-b391-6c4496bd0bec-webhook-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:21:13.397504 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.397474    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfdgs\" (UniqueName: \"kubernetes.io/projected/614c1594-2930-4926-b391-6c4496bd0bec-kube-api-access-gfdgs\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:21:13.879195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.879135    2564 generic.go:356] "Generic (PLEG): container finished" podID="614c1594-2930-4926-b391-6c4496bd0bec" containerID="4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa" exitCode=0
Jul 29 01:21:13.879619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.879205    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54666f4746-wps7x"
Jul 29 01:21:13.879619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.879224    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54666f4746-wps7x" event={"ID":"614c1594-2930-4926-b391-6c4496bd0bec","Type":"ContainerDied","Data":"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"}
Jul 29 01:21:13.879619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.879293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54666f4746-wps7x" event={"ID":"614c1594-2930-4926-b391-6c4496bd0bec","Type":"ContainerDied","Data":"054d3a4ba0cd76884859dc50874bb07dac03ebba784e2adba84bb97f25066536"}
Jul 29 01:21:13.879619 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.879328    2564 scope.go:117] "RemoveContainer" containerID="4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"
Jul 29 01:21:13.888885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.888871    2564 scope.go:117] "RemoveContainer" containerID="4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"
Jul 29 01:21:13.889118 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:21:13.889098    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa\": container with ID starting with 4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa not found: ID does not exist" containerID="4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"
Jul 29 01:21:13.889195 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.889127    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa"} err="failed to get container status \"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa\": rpc error: code = NotFound desc = could not find container \"4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa\": container with ID starting with 4fba630bd252082a447fb3f21a8f513498fc9d5e18a601d3028ca1cb40b9fdaa not found: ID does not exist"
Jul 29 01:21:13.910937 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.910917    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:21:13.920292 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:13.920274    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54666f4746-wps7x"]
Jul 29 01:21:14.117043 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:14.117006    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614c1594-2930-4926-b391-6c4496bd0bec" path="/var/lib/kubelet/pods/614c1594-2930-4926-b391-6c4496bd0bec/volumes"
Jul 29 01:21:32.621247 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.621211    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"]
Jul 29 01:21:32.621691 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.621616    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c1594-2930-4926-b391-6c4496bd0bec" containerName="manager"
Jul 29 01:21:32.621691 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.621627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c1594-2930-4926-b391-6c4496bd0bec" containerName="manager"
Jul 29 01:21:32.621764 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.621700    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="614c1594-2930-4926-b391-6c4496bd0bec" containerName="manager"
Jul 29 01:21:32.625962 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.625940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.628826 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.628805    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 01:21:32.628995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.628836    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h6hqw\""
Jul 29 01:21:32.637770 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.637746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"]
Jul 29 01:21:32.755697 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.755667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-maas-api-tls\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.755831 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.755728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77brh\" (UniqueName: \"kubernetes.io/projected/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-kube-api-access-77brh\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.755831 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.755807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-openshift-service-ca\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.856947 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.856912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-maas-api-tls\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.857087 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.856981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77brh\" (UniqueName: \"kubernetes.io/projected/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-kube-api-access-77brh\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.857087 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.857032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-openshift-service-ca\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.857687 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.857664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-openshift-service-ca\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.859398 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.859376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-maas-api-tls\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.880912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.880858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77brh\" (UniqueName: \"kubernetes.io/projected/01c33bff-bf76-4e27-8fd2-b1b40dde0b85-kube-api-access-77brh\") pod \"maas-api-74c9c4b7c9-s9vtt\" (UID: \"01c33bff-bf76-4e27-8fd2-b1b40dde0b85\") " pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:32.938004 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:32.937977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:33.101951 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:33.101929    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"]
Jul 29 01:21:33.104159 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:21:33.104118    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c33bff_bf76_4e27_8fd2_b1b40dde0b85.slice/crio-956c5e7d76c222a2e2d3309040dce799a1fec6ba6a429659974f21e525ff80b4 WatchSource:0}: Error finding container 956c5e7d76c222a2e2d3309040dce799a1fec6ba6a429659974f21e525ff80b4: Status 404 returned error can't find the container with id 956c5e7d76c222a2e2d3309040dce799a1fec6ba6a429659974f21e525ff80b4
Jul 29 01:21:34.005055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:34.005007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt" event={"ID":"01c33bff-bf76-4e27-8fd2-b1b40dde0b85","Type":"ContainerStarted","Data":"956c5e7d76c222a2e2d3309040dce799a1fec6ba6a429659974f21e525ff80b4"}
Jul 29 01:21:36.021789 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:36.021751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt" event={"ID":"01c33bff-bf76-4e27-8fd2-b1b40dde0b85","Type":"ContainerStarted","Data":"7c37baafeb4fe502277e755bd57f0587a11b012edee4374ff0c968e23423f412"}
Jul 29 01:21:36.022194 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:36.021898    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:36.043465 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:36.043421    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt" podStartSLOduration=2.103180346 podStartE2EDuration="4.043403004s" totalImagesPullingTime="1.940222658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:21:36.040906797 +0000 UTC m=+550.472789278" watchObservedRunningTime="2026-07-29 01:21:36.043403004 +0000 UTC m=+550.475285485"
Jul 29 01:21:41.061511 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.061478    2564 generic.go:356] "Generic (PLEG): container finished" podID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerID="98f52b4d0fa79b5cc7dc9632a921dd218f3a882d439258561b59f337e047cb56" exitCode=0
Jul 29 01:21:41.061924 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.061556    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" event={"ID":"95e13f08-b9f7-4782-84bd-0b62d4a33f76","Type":"ContainerDied","Data":"98f52b4d0fa79b5cc7dc9632a921dd218f3a882d439258561b59f337e047cb56"}
Jul 29 01:21:41.061968 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.061940    2564 scope.go:117] "RemoveContainer" containerID="98f52b4d0fa79b5cc7dc9632a921dd218f3a882d439258561b59f337e047cb56"
Jul 29 01:21:41.062794 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.062768    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:21:41.062891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.062849    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:21:41.062953 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:41.062901    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:21:42.032631 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:42.032602    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74c9c4b7c9-s9vtt"
Jul 29 01:21:42.071986 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:42.071952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" event={"ID":"95e13f08-b9f7-4782-84bd-0b62d4a33f76","Type":"ContainerStarted","Data":"7e3678aff87923f86cddefcedee0f2f0bee693ff3b8e8fa46fc2b792c2894083"}
Jul 29 01:21:42.254138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:42.254104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:21:42.255309 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:42.255280    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck9xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 29 01:21:42.255436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:42.255332    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podUID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 29 01:21:43.254013 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:43.253976    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck9xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 29 01:21:43.254423 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:43.254030    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podUID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 29 01:21:44.254328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:44.254290    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck9xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 29 01:21:44.254732 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:44.254382    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podUID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 29 01:21:45.254141 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:45.254103    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck9xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 29 01:21:45.254324 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:45.254196    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podUID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 29 01:21:46.254926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:46.254893    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck9xh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 29 01:21:46.255335 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:46.254953    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh" podUID="95e13f08-b9f7-4782-84bd-0b62d4a33f76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 29 01:21:46.493935 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:46.493900    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:21:47.257949 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:47.257920    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:21:47.258922 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:47.258904    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck9xh"
Jul 29 01:21:54.646815 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.646780    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"]
Jul 29 01:21:54.650265 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.650248    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.653164 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.653125    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-42t29\""
Jul 29 01:21:54.653282 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.653170    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 01:21:54.654499 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.654475    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 01:21:54.654576 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.654516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 01:21:54.661717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.661677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"]
Jul 29 01:21:54.742230 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.742346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.742346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.742420 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.742420 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703d8181-4a06-42d3-87d0-7f9d43c018a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.742488 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.742434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c652q\" (UniqueName: \"kubernetes.io/projected/703d8181-4a06-42d3-87d0-7f9d43c018a3-kube-api-access-c652q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843232 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703d8181-4a06-42d3-87d0-7f9d43c018a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843411 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c652q\" (UniqueName: \"kubernetes.io/projected/703d8181-4a06-42d3-87d0-7f9d43c018a3-kube-api-access-c652q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843696 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843619    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.843850 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.843818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.845571 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.845543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703d8181-4a06-42d3-87d0-7f9d43c018a3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.845830 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.845812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703d8181-4a06-42d3-87d0-7f9d43c018a3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.852905 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.852882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c652q\" (UniqueName: \"kubernetes.io/projected/703d8181-4a06-42d3-87d0-7f9d43c018a3-kube-api-access-c652q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq\" (UID: \"703d8181-4a06-42d3-87d0-7f9d43c018a3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:54.962371 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:54.962314    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:21:55.124663 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:55.124636    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"]
Jul 29 01:21:55.127260 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:21:55.127229    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703d8181_4a06_42d3_87d0_7f9d43c018a3.slice/crio-80a4198fb353ecba09c0fd72edf875bd4b7835edd1cb003346b2dea2c2b4b8cb WatchSource:0}: Error finding container 80a4198fb353ecba09c0fd72edf875bd4b7835edd1cb003346b2dea2c2b4b8cb: Status 404 returned error can't find the container with id 80a4198fb353ecba09c0fd72edf875bd4b7835edd1cb003346b2dea2c2b4b8cb
Jul 29 01:21:55.153448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:55.153416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq" event={"ID":"703d8181-4a06-42d3-87d0-7f9d43c018a3","Type":"ContainerStarted","Data":"80a4198fb353ecba09c0fd72edf875bd4b7835edd1cb003346b2dea2c2b4b8cb"}
Jul 29 01:21:57.167169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:57.167110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq" event={"ID":"703d8181-4a06-42d3-87d0-7f9d43c018a3","Type":"ContainerStarted","Data":"eeed5c93b086707aaa0298996f7e57b512b02e64e4e2c7af4a641e3af0ea0cfe"}
Jul 29 01:21:57.186368 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:21:57.186321    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq" podStartSLOduration=2.006178372 podStartE2EDuration="3.186306119s" totalImagesPullingTime="1.180127747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:21:57.184070855 +0000 UTC m=+571.615953334" watchObservedRunningTime="2026-07-29 01:21:57.186306119 +0000 UTC m=+571.618188611"
Jul 29 01:22:04.963418 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:04.963372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:22:04.963918 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:04.963449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:22:04.975677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:04.975654    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:22:05.222190 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:05.222094    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq"
Jul 29 01:22:26.004671 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:26.004641    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:22:26.006895 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:26.006873    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:22:42.324061 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.324025    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"]
Jul 29 01:22:42.329085 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.329065    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.332164 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.332124    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 01:22:42.341045 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.341017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"]
Jul 29 01:22:42.442030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.441993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.442259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.442051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.442259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.442120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvjs\" (UniqueName: \"kubernetes.io/projected/67d4d7f1-e32e-4326-9979-84754e16b460-kube-api-access-shvjs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.442259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.442182    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.442259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.442250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d4d7f1-e32e-4326-9979-84754e16b460-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.442452 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.442292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.542750 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.542904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.542904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.542904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvjs\" (UniqueName: \"kubernetes.io/projected/67d4d7f1-e32e-4326-9979-84754e16b460-kube-api-access-shvjs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.542904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.543102 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.542952    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d4d7f1-e32e-4326-9979-84754e16b460-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.543221 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.543199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.543291 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.543254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.543349 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.543308    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.545359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.545337    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d4d7f1-e32e-4326-9979-84754e16b460-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.545547 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.545531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d4d7f1-e32e-4326-9979-84754e16b460-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.551239 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.551220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvjs\" (UniqueName: \"kubernetes.io/projected/67d4d7f1-e32e-4326-9979-84754e16b460-kube-api-access-shvjs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp\" (UID: \"67d4d7f1-e32e-4326-9979-84754e16b460\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.648733 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.648668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:42.791287 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.791261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"]
Jul 29 01:22:42.791670 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:22:42.791643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d4d7f1_e32e_4326_9979_84754e16b460.slice/crio-ff044a29e3c47bdfb78bbfad8fde4e3a816e501fe1a8f6eda87c914c696534c4 WatchSource:0}: Error finding container ff044a29e3c47bdfb78bbfad8fde4e3a816e501fe1a8f6eda87c914c696534c4: Status 404 returned error can't find the container with id ff044a29e3c47bdfb78bbfad8fde4e3a816e501fe1a8f6eda87c914c696534c4
Jul 29 01:22:42.793573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:42.793557    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:22:43.435204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:43.435165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp" event={"ID":"67d4d7f1-e32e-4326-9979-84754e16b460","Type":"ContainerStarted","Data":"72e98712228e00370e7d81a1490bdcad5ea97cddfbb54416452112aec006a2ed"}
Jul 29 01:22:43.435630 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:43.435233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp" event={"ID":"67d4d7f1-e32e-4326-9979-84754e16b460","Type":"ContainerStarted","Data":"ff044a29e3c47bdfb78bbfad8fde4e3a816e501fe1a8f6eda87c914c696534c4"}
Jul 29 01:22:43.454789 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:43.454744    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp" podStartSLOduration=1.277532921 podStartE2EDuration="1.454730421s" totalImagesPullingTime="177.1975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:22:43.452055646 +0000 UTC m=+617.883938137" watchObservedRunningTime="2026-07-29 01:22:43.454730421 +0000 UTC m=+617.886612901"
Jul 29 01:22:52.649294 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:52.649207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:52.649294 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:52.649280    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:52.662028 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:52.661997    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:22:53.508400 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:22:53.508371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp"
Jul 29 01:24:01.766460 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.766425    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-648f9dbf77-rc9n2"]
Jul 29 01:24:01.770448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.770429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.775718 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.775692    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-648f9dbf77-rc9n2"]
Jul 29 01:24:01.804517 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.804493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ce1576e-6034-483c-89b2-9ec23dd55f59-tls-cert\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.804629 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.804606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bjq\" (UniqueName: \"kubernetes.io/projected/3ce1576e-6034-483c-89b2-9ec23dd55f59-kube-api-access-c5bjq\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.905020 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.904993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bjq\" (UniqueName: \"kubernetes.io/projected/3ce1576e-6034-483c-89b2-9ec23dd55f59-kube-api-access-c5bjq\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.905128 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.905035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ce1576e-6034-483c-89b2-9ec23dd55f59-tls-cert\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.907410 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.907390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ce1576e-6034-483c-89b2-9ec23dd55f59-tls-cert\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:01.913309 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:01.913282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bjq\" (UniqueName: \"kubernetes.io/projected/3ce1576e-6034-483c-89b2-9ec23dd55f59-kube-api-access-c5bjq\") pod \"authorino-648f9dbf77-rc9n2\" (UID: \"3ce1576e-6034-483c-89b2-9ec23dd55f59\") " pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:02.082220 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.082187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648f9dbf77-rc9n2"
Jul 29 01:24:02.220732 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.220706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-648f9dbf77-rc9n2"]
Jul 29 01:24:02.222077 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:24:02.222051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ce1576e_6034_483c_89b2_9ec23dd55f59.slice/crio-0bf390e5c00250651be009d5d9628ee0622e63f5853bb36b3507ee38a78eae77 WatchSource:0}: Error finding container 0bf390e5c00250651be009d5d9628ee0622e63f5853bb36b3507ee38a78eae77: Status 404 returned error can't find the container with id 0bf390e5c00250651be009d5d9628ee0622e63f5853bb36b3507ee38a78eae77
Jul 29 01:24:02.907370 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.907325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-648f9dbf77-rc9n2" event={"ID":"3ce1576e-6034-483c-89b2-9ec23dd55f59","Type":"ContainerStarted","Data":"0191698e3f62940a71b88d928a46a949b089100c42930457fc05632b444e3ea3"}
Jul 29 01:24:02.907856 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.907383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-648f9dbf77-rc9n2" event={"ID":"3ce1576e-6034-483c-89b2-9ec23dd55f59","Type":"ContainerStarted","Data":"0bf390e5c00250651be009d5d9628ee0622e63f5853bb36b3507ee38a78eae77"}
Jul 29 01:24:02.926638 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.926533    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-648f9dbf77-rc9n2" podStartSLOduration=1.4795325209999999 podStartE2EDuration="1.926511968s" totalImagesPullingTime="446.979447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:24:02.92379404 +0000 UTC m=+697.355676515" watchObservedRunningTime="2026-07-29 01:24:02.926511968 +0000 UTC m=+697.358394449"
Jul 29 01:24:02.955298 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.955263    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:24:02.955516 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:02.955482    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d88648b44-qcl7m" podUID="631b4472-2d90-44cd-8ce5-793d7fabbea1" containerName="authorino" containerID="cri-o://da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311" gracePeriod=30
Jul 29 01:24:03.239811 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.239781    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:24:03.323341 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.323308    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert\") pod \"631b4472-2d90-44cd-8ce5-793d7fabbea1\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") "
Jul 29 01:24:03.323532 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.323387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf889\" (UniqueName: \"kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889\") pod \"631b4472-2d90-44cd-8ce5-793d7fabbea1\" (UID: \"631b4472-2d90-44cd-8ce5-793d7fabbea1\") "
Jul 29 01:24:03.325428 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.325390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889" (OuterVolumeSpecName: "kube-api-access-pf889") pod "631b4472-2d90-44cd-8ce5-793d7fabbea1" (UID: "631b4472-2d90-44cd-8ce5-793d7fabbea1"). InnerVolumeSpecName "kube-api-access-pf889". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:24:03.334727 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.334700    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "631b4472-2d90-44cd-8ce5-793d7fabbea1" (UID: "631b4472-2d90-44cd-8ce5-793d7fabbea1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:24:03.424501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.424468    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf889\" (UniqueName: \"kubernetes.io/projected/631b4472-2d90-44cd-8ce5-793d7fabbea1-kube-api-access-pf889\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:24:03.424501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.424493    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/631b4472-2d90-44cd-8ce5-793d7fabbea1-tls-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:24:03.914846 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.914811    2564 generic.go:356] "Generic (PLEG): container finished" podID="631b4472-2d90-44cd-8ce5-793d7fabbea1" containerID="da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311" exitCode=0
Jul 29 01:24:03.915219 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.914883    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d88648b44-qcl7m"
Jul 29 01:24:03.915219 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.914892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d88648b44-qcl7m" event={"ID":"631b4472-2d90-44cd-8ce5-793d7fabbea1","Type":"ContainerDied","Data":"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"}
Jul 29 01:24:03.915219 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.914945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d88648b44-qcl7m" event={"ID":"631b4472-2d90-44cd-8ce5-793d7fabbea1","Type":"ContainerDied","Data":"6aa3b550464137de615bfd95b41faac196d245c5e7e928eb30fd7d6198875520"}
Jul 29 01:24:03.915219 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.914969    2564 scope.go:117] "RemoveContainer" containerID="da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"
Jul 29 01:24:03.927552 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.927377    2564 scope.go:117] "RemoveContainer" containerID="da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"
Jul 29 01:24:03.927749 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:24:03.927717    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311\": container with ID starting with da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311 not found: ID does not exist" containerID="da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"
Jul 29 01:24:03.927842 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.927753    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311"} err="failed to get container status \"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311\": rpc error: code = NotFound desc = could not find container \"da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311\": container with ID starting with da4ede14972d0e4422034b75575f9cbccb01754718d042a6caf85a9e0912f311 not found: ID does not exist"
Jul 29 01:24:03.945549 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.945521    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:24:03.955878 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:03.955854    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d88648b44-qcl7m"]
Jul 29 01:24:04.116833 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:04.116799    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="631b4472-2d90-44cd-8ce5-793d7fabbea1" path="/var/lib/kubelet/pods/631b4472-2d90-44cd-8ce5-793d7fabbea1/volumes"
Jul 29 01:24:49.026917 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.026880    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:24:49.027428 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.027113    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6dd887db67-pzlbp" podUID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" containerName="manager" containerID="cri-o://d67169d0ca9fa9e89a3aa812c036b7f6d2a33f3819f7e7e9009b89e8820f8a9e" gracePeriod=30
Jul 29 01:24:49.185950 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.185913    2564 generic.go:356] "Generic (PLEG): container finished" podID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" containerID="d67169d0ca9fa9e89a3aa812c036b7f6d2a33f3819f7e7e9009b89e8820f8a9e" exitCode=0
Jul 29 01:24:49.186104 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.185990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dd887db67-pzlbp" event={"ID":"401d870b-0699-4a36-9dc7-8e5ffdfe5e03","Type":"ContainerDied","Data":"d67169d0ca9fa9e89a3aa812c036b7f6d2a33f3819f7e7e9009b89e8820f8a9e"}
Jul 29 01:24:49.284623 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.284567    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:24:49.336671 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.336642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert\") pod \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") "
Jul 29 01:24:49.336797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.336779    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gbgb\" (UniqueName: \"kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb\") pod \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\" (UID: \"401d870b-0699-4a36-9dc7-8e5ffdfe5e03\") "
Jul 29 01:24:49.338633 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.338607    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb" (OuterVolumeSpecName: "kube-api-access-9gbgb") pod "401d870b-0699-4a36-9dc7-8e5ffdfe5e03" (UID: "401d870b-0699-4a36-9dc7-8e5ffdfe5e03"). InnerVolumeSpecName "kube-api-access-9gbgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:24:49.338796 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.338777    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "401d870b-0699-4a36-9dc7-8e5ffdfe5e03" (UID: "401d870b-0699-4a36-9dc7-8e5ffdfe5e03"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:24:49.437683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.437646    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-webhook-cert\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:24:49.437683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:49.437682    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gbgb\" (UniqueName: \"kubernetes.io/projected/401d870b-0699-4a36-9dc7-8e5ffdfe5e03-kube-api-access-9gbgb\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:24:50.192828 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:50.192804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dd887db67-pzlbp"
Jul 29 01:24:50.193227 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:50.192847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dd887db67-pzlbp" event={"ID":"401d870b-0699-4a36-9dc7-8e5ffdfe5e03","Type":"ContainerDied","Data":"3c60bf6a481013be09d9a6e3751965c53fe021e785c5e366e5b716226922e9ac"}
Jul 29 01:24:50.193227 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:50.192908    2564 scope.go:117] "RemoveContainer" containerID="d67169d0ca9fa9e89a3aa812c036b7f6d2a33f3819f7e7e9009b89e8820f8a9e"
Jul 29 01:24:50.220890 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:50.220868    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:24:50.223935 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:50.223914    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6dd887db67-pzlbp"]
Jul 29 01:24:52.117259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:24:52.117218    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" path="/var/lib/kubelet/pods/401d870b-0699-4a36-9dc7-8e5ffdfe5e03/volumes"
Jul 29 01:27:26.046121 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:27:26.046042    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:27:26.046957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:27:26.046934    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:30:00.147054 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147021    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"]
Jul 29 01:30:00.147662 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147640    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" containerName="manager"
Jul 29 01:30:00.147736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147662    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" containerName="manager"
Jul 29 01:30:00.147736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147686    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631b4472-2d90-44cd-8ce5-793d7fabbea1" containerName="authorino"
Jul 29 01:30:00.147736 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147696    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="631b4472-2d90-44cd-8ce5-793d7fabbea1" containerName="authorino"
Jul 29 01:30:00.147884 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147817    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="401d870b-0699-4a36-9dc7-8e5ffdfe5e03" containerName="manager"
Jul 29 01:30:00.147884 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.147835    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="631b4472-2d90-44cd-8ce5-793d7fabbea1" containerName="authorino"
Jul 29 01:30:00.150882 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.150860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"
Jul 29 01:30:00.153621 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.153598    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:30:00.160746 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.160725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"
Jul 29 01:30:00.167055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.167031    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"]
Jul 29 01:30:00.340874 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.340844    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"]
Jul 29 01:30:00.342633 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:30:00.342608    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c80dc1_8bbc_4f4a_89f6_03f0ecdd669f.slice/crio-25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b WatchSource:0}: Error finding container 25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b: Status 404 returned error can't find the container with id 25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b
Jul 29 01:30:00.344712 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.344684    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:30:00.995766 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:00.995725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558" event={"ID":"89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f","Type":"ContainerStarted","Data":"25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b"}
Jul 29 01:30:04.015845 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:04.015764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558" event={"ID":"89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f","Type":"ContainerStarted","Data":"e04f453adbc0e61ec59fbe611a241ce141db2268bc244784fad0404be2630d70"}
Jul 29 01:30:04.032531 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:04.032480    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558" podStartSLOduration=0.967246976 podStartE2EDuration="4.032465899s" totalImagesPullingTime="3.065218923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:30:04.030497208 +0000 UTC m=+1058.462379689" watchObservedRunningTime="2026-07-29 01:30:04.032465899 +0000 UTC m=+1058.464348379"
Jul 29 01:30:05.023171 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:05.023107    2564 generic.go:356] "Generic (PLEG): container finished" podID="89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f" containerID="e04f453adbc0e61ec59fbe611a241ce141db2268bc244784fad0404be2630d70" exitCode=0
Jul 29 01:30:05.023572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:05.023193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558" event={"ID":"89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f","Type":"ContainerDied","Data":"e04f453adbc0e61ec59fbe611a241ce141db2268bc244784fad0404be2630d70"}
Jul 29 01:30:06.159636 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:06.159614    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"
Jul 29 01:30:07.038098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:07.038069    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558"
Jul 29 01:30:07.038098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:07.038079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754810-s9558" event={"ID":"89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f","Type":"ContainerDied","Data":"25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b"}
Jul 29 01:30:07.038343 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:30:07.038129    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25a4930760e2a7b5aef0818335c02a406fcddeda2de5a79855bf725ae3a7661b"
Jul 29 01:31:02.798800 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:02.798764    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:31:02.799404 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:02.798971    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" podUID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" containerName="manager" containerID="cri-o://188c190156ea5d862ccd2fe7acab8ca8bf59a74f8a957a62eb549968743b7b88" gracePeriod=10
Jul 29 01:31:06.390080 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.390043    2564 generic.go:356] "Generic (PLEG): container finished" podID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" containerID="188c190156ea5d862ccd2fe7acab8ca8bf59a74f8a957a62eb549968743b7b88" exitCode=0
Jul 29 01:31:06.390472 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.390119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" event={"ID":"113f68bb-5b88-47b5-9bf7-c37eb9543f8f","Type":"ContainerDied","Data":"188c190156ea5d862ccd2fe7acab8ca8bf59a74f8a957a62eb549968743b7b88"}
Jul 29 01:31:06.468060 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.468040    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:31:06.590565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.590489    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume\") pod \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") "
Jul 29 01:31:06.590565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.590561    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc5l9\" (UniqueName: \"kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9\") pod \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\" (UID: \"113f68bb-5b88-47b5-9bf7-c37eb9543f8f\") "
Jul 29 01:31:06.590841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.590819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "113f68bb-5b88-47b5-9bf7-c37eb9543f8f" (UID: "113f68bb-5b88-47b5-9bf7-c37eb9543f8f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:31:06.592546 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.592526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9" (OuterVolumeSpecName: "kube-api-access-wc5l9") pod "113f68bb-5b88-47b5-9bf7-c37eb9543f8f" (UID: "113f68bb-5b88-47b5-9bf7-c37eb9543f8f"). InnerVolumeSpecName "kube-api-access-wc5l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:31:06.691852 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.691829    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc5l9\" (UniqueName: \"kubernetes.io/projected/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-kube-api-access-wc5l9\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:31:06.691852 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:06.691852    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/113f68bb-5b88-47b5-9bf7-c37eb9543f8f-extensions-socket-volume\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:31:07.397288 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:07.397261    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"
Jul 29 01:31:07.397732 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:07.397295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw" event={"ID":"113f68bb-5b88-47b5-9bf7-c37eb9543f8f","Type":"ContainerDied","Data":"b73169f9654f50de5ff090e5e95f0e20a2425144802b4067ec97e1064e598e1c"}
Jul 29 01:31:07.397732 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:07.397359    2564 scope.go:117] "RemoveContainer" containerID="188c190156ea5d862ccd2fe7acab8ca8bf59a74f8a957a62eb549968743b7b88"
Jul 29 01:31:07.421844 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:07.421814    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:31:07.425534 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:07.425495    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kjsnw"]
Jul 29 01:31:08.117342 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:31:08.117310    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" path="/var/lib/kubelet/pods/113f68bb-5b88-47b5-9bf7-c37eb9543f8f/volumes"
Jul 29 01:32:03.897567 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.897520    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"]
Jul 29 01:32:03.899223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899105    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f" containerName="cleanup"
Jul 29 01:32:03.901271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f" containerName="cleanup"
Jul 29 01:32:03.901271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899318    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" containerName="manager"
Jul 29 01:32:03.901271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899328    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" containerName="manager"
Jul 29 01:32:03.901271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899430    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="113f68bb-5b88-47b5-9bf7-c37eb9543f8f" containerName="manager"
Jul 29 01:32:03.901271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.899442    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f" containerName="cleanup"
Jul 29 01:32:03.901706 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.901689    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:03.904724 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.904701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-77d5k\""
Jul 29 01:32:03.913008 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.912987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"]
Jul 29 01:32:03.955185 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.955140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9xl\" (UniqueName: \"kubernetes.io/projected/bf27adbd-5850-43a5-a34d-bc96fceecd82-kube-api-access-2q9xl\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:03.955314 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:03.955290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf27adbd-5850-43a5-a34d-bc96fceecd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.056326 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.056298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf27adbd-5850-43a5-a34d-bc96fceecd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.056431 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.056365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9xl\" (UniqueName: \"kubernetes.io/projected/bf27adbd-5850-43a5-a34d-bc96fceecd82-kube-api-access-2q9xl\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.056691 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.056671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf27adbd-5850-43a5-a34d-bc96fceecd82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.064557 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.064529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9xl\" (UniqueName: \"kubernetes.io/projected/bf27adbd-5850-43a5-a34d-bc96fceecd82-kube-api-access-2q9xl\") pod \"kuadrant-operator-controller-manager-86878c769f-hlsrt\" (UID: \"bf27adbd-5850-43a5-a34d-bc96fceecd82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.213513 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.213445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.362403 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:32:04.362373    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf27adbd_5850_43a5_a34d_bc96fceecd82.slice/crio-737e383ea705931e09ea2f87fe7b1b1e629f9d266d0899bc034a54db4ab3ec47 WatchSource:0}: Error finding container 737e383ea705931e09ea2f87fe7b1b1e629f9d266d0899bc034a54db4ab3ec47: Status 404 returned error can't find the container with id 737e383ea705931e09ea2f87fe7b1b1e629f9d266d0899bc034a54db4ab3ec47
Jul 29 01:32:04.365331 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.365307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"]
Jul 29 01:32:04.733788 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.733752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt" event={"ID":"bf27adbd-5850-43a5-a34d-bc96fceecd82","Type":"ContainerStarted","Data":"077c790cfd842cda2a6125f8422e7e425d738df24107e9952775236f8a7c26f9"}
Jul 29 01:32:04.733978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.733808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt" event={"ID":"bf27adbd-5850-43a5-a34d-bc96fceecd82","Type":"ContainerStarted","Data":"737e383ea705931e09ea2f87fe7b1b1e629f9d266d0899bc034a54db4ab3ec47"}
Jul 29 01:32:04.733978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.733831    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:04.757594 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:04.757548    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt" podStartSLOduration=1.7575353 podStartE2EDuration="1.7575353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:32:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:32:04.754353572 +0000 UTC m=+1179.186236065" watchObservedRunningTime="2026-07-29 01:32:04.7575353 +0000 UTC m=+1179.189417780"
Jul 29 01:32:15.743573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:15.743544    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hlsrt"
Jul 29 01:32:26.092662 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:26.092627    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:32:26.095005 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:32:26.094979    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:37:26.139558 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:37:26.139530    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:37:26.146740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:37:26.146717    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:40:08.171469 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.171435    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:08.174034 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.174011    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.176904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.176877    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ed6c6ab5-serving-cert\""
Jul 29 01:40:08.188083 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.188060    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:08.220379 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.220354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.220466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.220383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.220466 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.220458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87km8\" (UniqueName: \"kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.320922 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.320886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.320922 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.320924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.321140 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.321056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87km8\" (UniqueName: \"kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.321690 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.321670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.323319 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.323295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.338912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.338889    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87km8\" (UniqueName: \"kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8\") pod \"maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.484931 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.484855    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:08.625892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.625862    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:08.626923 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:40:08.626897    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1dbac01_9805_4976_9be2_5e0a88950024.slice/crio-b8f0fcd1330ea742cea23b1ea595a688194ae4ec135e2eb39308f93cce68a09d WatchSource:0}: Error finding container b8f0fcd1330ea742cea23b1ea595a688194ae4ec135e2eb39308f93cce68a09d: Status 404 returned error can't find the container with id b8f0fcd1330ea742cea23b1ea595a688194ae4ec135e2eb39308f93cce68a09d
Jul 29 01:40:08.628280 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:08.628261    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:40:09.574289 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:09.574238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" event={"ID":"b1dbac01-9805-4976-9be2-5e0a88950024","Type":"ContainerStarted","Data":"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"}
Jul 29 01:40:09.574679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:09.574308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" event={"ID":"b1dbac01-9805-4976-9be2-5e0a88950024","Type":"ContainerStarted","Data":"b8f0fcd1330ea742cea23b1ea595a688194ae4ec135e2eb39308f93cce68a09d"}
Jul 29 01:40:09.574679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:09.574392    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:09.595723 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:09.595676    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" podStartSLOduration=1.089303635 podStartE2EDuration="1.595662016s" totalImagesPullingTime="506.358381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:40:09.594037987 +0000 UTC m=+1664.025920466" watchObservedRunningTime="2026-07-29 01:40:09.595662016 +0000 UTC m=+1664.027544538"
Jul 29 01:40:15.584978 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:15.584945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:33.673047 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.673013    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"]
Jul 29 01:40:33.675758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.675742    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:33.678526 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.678504    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:40:33.683980 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.683955    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"]
Jul 29 01:40:33.725199 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.725171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b\" (UID: \"f4146898-8a52-46f1-a9c3-06e232924aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:33.825665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.825637    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b\" (UID: \"f4146898-8a52-46f1-a9c3-06e232924aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:33.826273 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.826250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b\" (UID: \"f4146898-8a52-46f1-a9c3-06e232924aab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:33.987191 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:33.987115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:34.121380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:34.121349    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"]
Jul 29 01:40:34.121843 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:40:34.121811    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4146898_8a52_46f1_a9c3_06e232924aab.slice/crio-f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e WatchSource:0}: Error finding container f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e: Status 404 returned error can't find the container with id f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e
Jul 29 01:40:34.721741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:34.721663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b" event={"ID":"f4146898-8a52-46f1-a9c3-06e232924aab","Type":"ContainerStarted","Data":"dece9e98cb74d27a96e78ad9a09e761198cdca295b90c8cb363c581cd3a930fc"}
Jul 29 01:40:34.721741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:34.721712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b" event={"ID":"f4146898-8a52-46f1-a9c3-06e232924aab","Type":"ContainerStarted","Data":"f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e"}
Jul 29 01:40:34.739568 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:34.739522    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b" podStartSLOduration=1.7395070910000001 podStartE2EDuration="1.739507091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:40:34.736469113 +0000 UTC m=+1689.168351596" watchObservedRunningTime="2026-07-29 01:40:34.739507091 +0000 UTC m=+1689.171389570"
Jul 29 01:40:35.730131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:35.730098    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4146898-8a52-46f1-a9c3-06e232924aab" containerID="dece9e98cb74d27a96e78ad9a09e761198cdca295b90c8cb363c581cd3a930fc" exitCode=0
Jul 29 01:40:35.730514 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:35.730174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b" event={"ID":"f4146898-8a52-46f1-a9c3-06e232924aab","Type":"ContainerDied","Data":"dece9e98cb74d27a96e78ad9a09e761198cdca295b90c8cb363c581cd3a930fc"}
Jul 29 01:40:36.872435 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:36.872411    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:36.954220 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:36.954195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca\") pod \"f4146898-8a52-46f1-a9c3-06e232924aab\" (UID: \"f4146898-8a52-46f1-a9c3-06e232924aab\") "
Jul 29 01:40:36.954571 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:36.954553    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4146898-8a52-46f1-a9c3-06e232924aab" (UID: "f4146898-8a52-46f1-a9c3-06e232924aab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:40:37.055568 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:37.055541    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4146898-8a52-46f1-a9c3-06e232924aab-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:40:37.743165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:37.743116    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b" event={"ID":"f4146898-8a52-46f1-a9c3-06e232924aab","Type":"ContainerDied","Data":"f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e"}
Jul 29 01:40:37.743165 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:37.743143    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"
Jul 29 01:40:37.743380 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:37.743181    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f36d25abd2a2160729922b537669fbd89d11f743e80c086e2fc1fd3da224fd7e"
Jul 29 01:40:38.974841 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:38.974806    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:38.975445 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:38.975068    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" podUID="b1dbac01-9805-4976-9be2-5e0a88950024" containerName="maas-api" containerID="cri-o://b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a" gracePeriod=30
Jul 29 01:40:39.174910 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.174871    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"]
Jul 29 01:40:39.178963 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.178934    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed6c6ab5-8vp7b"]
Jul 29 01:40:39.248364 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.248343    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:39.373441 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.373404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls\") pod \"b1dbac01-9805-4976-9be2-5e0a88950024\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") "
Jul 29 01:40:39.373441 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.373444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87km8\" (UniqueName: \"kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8\") pod \"b1dbac01-9805-4976-9be2-5e0a88950024\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") "
Jul 29 01:40:39.373643 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.373530    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca\") pod \"b1dbac01-9805-4976-9be2-5e0a88950024\" (UID: \"b1dbac01-9805-4976-9be2-5e0a88950024\") "
Jul 29 01:40:39.373957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.373918    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1dbac01-9805-4976-9be2-5e0a88950024" (UID: "b1dbac01-9805-4976-9be2-5e0a88950024"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:40:39.375746 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.375714    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1dbac01-9805-4976-9be2-5e0a88950024" (UID: "b1dbac01-9805-4976-9be2-5e0a88950024"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:40:39.375846 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.375776    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8" (OuterVolumeSpecName: "kube-api-access-87km8") pod "b1dbac01-9805-4976-9be2-5e0a88950024" (UID: "b1dbac01-9805-4976-9be2-5e0a88950024"). InnerVolumeSpecName "kube-api-access-87km8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:40:39.474450 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.474419    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1dbac01-9805-4976-9be2-5e0a88950024-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:40:39.474450 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.474444    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1dbac01-9805-4976-9be2-5e0a88950024-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:40:39.474450 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.474455    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87km8\" (UniqueName: \"kubernetes.io/projected/b1dbac01-9805-4976-9be2-5e0a88950024-kube-api-access-87km8\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:40:39.754967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.754933    2564 generic.go:356] "Generic (PLEG): container finished" podID="b1dbac01-9805-4976-9be2-5e0a88950024" containerID="b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a" exitCode=0
Jul 29 01:40:39.754967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.754971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" event={"ID":"b1dbac01-9805-4976-9be2-5e0a88950024","Type":"ContainerDied","Data":"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"}
Jul 29 01:40:39.755169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.754974    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"
Jul 29 01:40:39.755169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.755013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch" event={"ID":"b1dbac01-9805-4976-9be2-5e0a88950024","Type":"ContainerDied","Data":"b8f0fcd1330ea742cea23b1ea595a688194ae4ec135e2eb39308f93cce68a09d"}
Jul 29 01:40:39.755169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.755036    2564 scope.go:117] "RemoveContainer" containerID="b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"
Jul 29 01:40:39.765432 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.765387    2564 scope.go:117] "RemoveContainer" containerID="b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"
Jul 29 01:40:39.765737 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:40:39.765702    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a\": container with ID starting with b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a not found: ID does not exist" containerID="b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"
Jul 29 01:40:39.765863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.765746    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a"} err="failed to get container status \"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a\": rpc error: code = NotFound desc = could not find container \"b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a\": container with ID starting with b627db673cd436c01f607a37a4b62eab76c7abdc929878dd7fc89c5914fb8b2a not found: ID does not exist"
Jul 29 01:40:39.795971 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.795938    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:39.803127 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:39.803105    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ed6c6ab5-5dbd465bf8-v69ch"]
Jul 29 01:40:40.117493 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:40.117461    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1dbac01-9805-4976-9be2-5e0a88950024" path="/var/lib/kubelet/pods/b1dbac01-9805-4976-9be2-5e0a88950024/volumes"
Jul 29 01:40:40.117860 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:40.117844    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4146898-8a52-46f1-a9c3-06e232924aab" path="/var/lib/kubelet/pods/f4146898-8a52-46f1-a9c3-06e232924aab/volumes"
Jul 29 01:40:43.460776 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.460745    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:40:43.461197 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461183    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4146898-8a52-46f1-a9c3-06e232924aab" containerName="revoke-keys"
Jul 29 01:40:43.461255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461198    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4146898-8a52-46f1-a9c3-06e232924aab" containerName="revoke-keys"
Jul 29 01:40:43.461255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461208    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1dbac01-9805-4976-9be2-5e0a88950024" containerName="maas-api"
Jul 29 01:40:43.461255 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461214    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1dbac01-9805-4976-9be2-5e0a88950024" containerName="maas-api"
Jul 29 01:40:43.461348 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461304    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1dbac01-9805-4976-9be2-5e0a88950024" containerName="maas-api"
Jul 29 01:40:43.461348 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.461316    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4146898-8a52-46f1-a9c3-06e232924aab" containerName="revoke-keys"
Jul 29 01:40:43.466049 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.465992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.468842 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.468824    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6870a9be-serving-cert\""
Jul 29 01:40:43.474651 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.474327    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:40:43.614833 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.614798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.615013 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.614864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.615013 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.614946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjj5s\" (UniqueName: \"kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.715568 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.715496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.715568 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.715556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.715766 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.715581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjj5s\" (UniqueName: \"kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.715766 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:40:43.715641    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-serving-cert: secret "maas-api-e2e-migrate-6870a9be-serving-cert" not found
Jul 29 01:40:43.715766 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:40:43.715703    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls podName:333a29f4-827a-46e7-bac5-4dfbaabd43b0 nodeName:}" failed. No retries permitted until 2026-07-29 01:40:44.215688089 +0000 UTC m=+1698.647570546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls") pod "maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" (UID: "333a29f4-827a-46e7-bac5-4dfbaabd43b0") : secret "maas-api-e2e-migrate-6870a9be-serving-cert" not found
Jul 29 01:40:43.716229 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.716211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:43.726398 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:43.726378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjj5s\" (UniqueName: \"kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:44.219601 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:44.219569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:44.221903 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:44.221885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") pod \"maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:44.378919 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:44.378884    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:44.522778 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:44.522752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:40:44.524527 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:40:44.524500    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333a29f4_827a_46e7_bac5_4dfbaabd43b0.slice/crio-abdba1ae0c977d61fd325e40a11c0ae4d146f223f930ac6a14a608e24f4c5f53 WatchSource:0}: Error finding container abdba1ae0c977d61fd325e40a11c0ae4d146f223f930ac6a14a608e24f4c5f53: Status 404 returned error can't find the container with id abdba1ae0c977d61fd325e40a11c0ae4d146f223f930ac6a14a608e24f4c5f53
Jul 29 01:40:44.788829 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:44.788797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" event={"ID":"333a29f4-827a-46e7-bac5-4dfbaabd43b0","Type":"ContainerStarted","Data":"abdba1ae0c977d61fd325e40a11c0ae4d146f223f930ac6a14a608e24f4c5f53"}
Jul 29 01:40:45.796404 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:45.796368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" event={"ID":"333a29f4-827a-46e7-bac5-4dfbaabd43b0","Type":"ContainerStarted","Data":"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"}
Jul 29 01:40:45.796756 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:45.796482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:40:45.814992 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:45.814946    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" podStartSLOduration=2.52988588 podStartE2EDuration="2.814930682s" totalImagesPullingTime="285.044802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:40:45.812693003 +0000 UTC m=+1700.244575502" watchObservedRunningTime="2026-07-29 01:40:45.814930682 +0000 UTC m=+1700.246813178"
Jul 29 01:40:51.808569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:40:51.808490    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:41:08.341487 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.341445    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"]
Jul 29 01:41:08.345177 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.345137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:08.348030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.348006    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:41:08.352497 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.352449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"]
Jul 29 01:41:08.438888 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.438862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6\" (UID: \"6aca416d-88b8-4829-ab52-cd46b59b6d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:08.539428 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.539403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6\" (UID: \"6aca416d-88b8-4829-ab52-cd46b59b6d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:08.540006 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.539984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6\" (UID: \"6aca416d-88b8-4829-ab52-cd46b59b6d60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:08.657637 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.657572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:08.994593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:08.994569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"]
Jul 29 01:41:08.996591 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:41:08.996561    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aca416d_88b8_4829_ab52_cd46b59b6d60.slice/crio-e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285 WatchSource:0}: Error finding container e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285: Status 404 returned error can't find the container with id e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285
Jul 29 01:41:09.942595 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:09.942518    2564 generic.go:356] "Generic (PLEG): container finished" podID="6aca416d-88b8-4829-ab52-cd46b59b6d60" containerID="86b91efc5bd5638e862a3b865b43554d9e389da03a6e7823baa95317aa5bc6e3" exitCode=0
Jul 29 01:41:09.942595 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:09.942575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6" event={"ID":"6aca416d-88b8-4829-ab52-cd46b59b6d60","Type":"ContainerDied","Data":"86b91efc5bd5638e862a3b865b43554d9e389da03a6e7823baa95317aa5bc6e3"}
Jul 29 01:41:09.942949 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:09.942611    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6" event={"ID":"6aca416d-88b8-4829-ab52-cd46b59b6d60","Type":"ContainerStarted","Data":"e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285"}
Jul 29 01:41:11.085573 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.085548    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:11.159339 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.159313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca\") pod \"6aca416d-88b8-4829-ab52-cd46b59b6d60\" (UID: \"6aca416d-88b8-4829-ab52-cd46b59b6d60\") "
Jul 29 01:41:11.159701 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.159680    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6aca416d-88b8-4829-ab52-cd46b59b6d60" (UID: "6aca416d-88b8-4829-ab52-cd46b59b6d60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:41:11.260169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.260099    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aca416d-88b8-4829-ab52-cd46b59b6d60-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:11.955892 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.955858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6" event={"ID":"6aca416d-88b8-4829-ab52-cd46b59b6d60","Type":"ContainerDied","Data":"e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285"}
Jul 29 01:41:11.956067 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.955907    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6c5a7d9ca869885bdbbcfc16272374b6865f8a01c82419655a7551f159d3285"
Jul 29 01:41:11.956067 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:11.955872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"
Jul 29 01:41:15.005663 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.005625    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:41:15.006203 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.006169    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" podUID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" containerName="maas-api" containerID="cri-o://334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d" gracePeriod=30
Jul 29 01:41:15.216838 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.216807    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"]
Jul 29 01:41:15.221226 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.221199    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6870a9be-5fpn6"]
Jul 29 01:41:15.266572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.266497    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:41:15.294251 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.294223    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca\") pod \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") "
Jul 29 01:41:15.294384 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.294353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") pod \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") "
Jul 29 01:41:15.294438 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.294423    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjj5s\" (UniqueName: \"kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s\") pod \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\" (UID: \"333a29f4-827a-46e7-bac5-4dfbaabd43b0\") "
Jul 29 01:41:15.294610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.294585    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "333a29f4-827a-46e7-bac5-4dfbaabd43b0" (UID: "333a29f4-827a-46e7-bac5-4dfbaabd43b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:41:15.294778 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.294757    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/333a29f4-827a-46e7-bac5-4dfbaabd43b0-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:15.296460 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.296431    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "333a29f4-827a-46e7-bac5-4dfbaabd43b0" (UID: "333a29f4-827a-46e7-bac5-4dfbaabd43b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:41:15.296460 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.296440    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s" (OuterVolumeSpecName: "kube-api-access-rjj5s") pod "333a29f4-827a-46e7-bac5-4dfbaabd43b0" (UID: "333a29f4-827a-46e7-bac5-4dfbaabd43b0"). InnerVolumeSpecName "kube-api-access-rjj5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:41:15.395557 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.395522    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/333a29f4-827a-46e7-bac5-4dfbaabd43b0-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:15.395557 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.395551    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjj5s\" (UniqueName: \"kubernetes.io/projected/333a29f4-827a-46e7-bac5-4dfbaabd43b0-kube-api-access-rjj5s\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:15.981565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.981528    2564 generic.go:356] "Generic (PLEG): container finished" podID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" containerID="334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d" exitCode=0
Jul 29 01:41:15.981758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.981575    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"
Jul 29 01:41:15.981758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.981616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" event={"ID":"333a29f4-827a-46e7-bac5-4dfbaabd43b0","Type":"ContainerDied","Data":"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"}
Jul 29 01:41:15.981758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.981673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l" event={"ID":"333a29f4-827a-46e7-bac5-4dfbaabd43b0","Type":"ContainerDied","Data":"abdba1ae0c977d61fd325e40a11c0ae4d146f223f930ac6a14a608e24f4c5f53"}
Jul 29 01:41:15.981758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.981697    2564 scope.go:117] "RemoveContainer" containerID="334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"
Jul 29 01:41:15.992735 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.992711    2564 scope.go:117] "RemoveContainer" containerID="334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"
Jul 29 01:41:15.993023 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:41:15.993004    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d\": container with ID starting with 334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d not found: ID does not exist" containerID="334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"
Jul 29 01:41:15.993099 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:15.993032    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d"} err="failed to get container status \"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d\": rpc error: code = NotFound desc = could not find container \"334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d\": container with ID starting with 334910b809c268374bad4980b79de5bf2575122cf4a9c93abfecc1dcc3f7bd2d not found: ID does not exist"
Jul 29 01:41:16.008120 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:16.008077    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:41:16.012185 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:16.012133    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6870a9be-84c69b6dfc-ggc9l"]
Jul 29 01:41:16.117703 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:16.117671    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" path="/var/lib/kubelet/pods/333a29f4-827a-46e7-bac5-4dfbaabd43b0/volumes"
Jul 29 01:41:16.118068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:16.118052    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aca416d-88b8-4829-ab52-cd46b59b6d60" path="/var/lib/kubelet/pods/6aca416d-88b8-4829-ab52-cd46b59b6d60/volumes"
Jul 29 01:41:18.615759 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.615720    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:18.616143 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616130    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aca416d-88b8-4829-ab52-cd46b59b6d60" containerName="revoke-keys"
Jul 29 01:41:18.616143 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616168    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aca416d-88b8-4829-ab52-cd46b59b6d60" containerName="revoke-keys"
Jul 29 01:41:18.616269 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616189    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" containerName="maas-api"
Jul 29 01:41:18.616269 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616194    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" containerName="maas-api"
Jul 29 01:41:18.616269 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616260    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aca416d-88b8-4829-ab52-cd46b59b6d60" containerName="revoke-keys"
Jul 29 01:41:18.616269 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.616270    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="333a29f4-827a-46e7-bac5-4dfbaabd43b0" containerName="maas-api"
Jul 29 01:41:18.620865 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.620847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.623753 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.623733    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a3670250-serving-cert\""
Jul 29 01:41:18.632256 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.632234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:18.726000 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.725958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.726187 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.726015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgpxj\" (UniqueName: \"kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.726187 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.726074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.827299 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.827260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.827487 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.827309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgpxj\" (UniqueName: \"kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.827487 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.827428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.827597 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:41:18.827572    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-serving-cert: secret "maas-api-e2e-ait-a3670250-serving-cert" not found
Jul 29 01:41:18.827661 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:41:18.827649    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls podName:5620aa55-d0e5-4fc1-b087-7fec29e81708 nodeName:}" failed. No retries permitted until 2026-07-29 01:41:19.32763133 +0000 UTC m=+1733.759513802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls") pod "maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" (UID: "5620aa55-d0e5-4fc1-b087-7fec29e81708") : secret "maas-api-e2e-ait-a3670250-serving-cert" not found
Jul 29 01:41:18.827875 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.827857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:18.836397 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:18.836374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgpxj\" (UniqueName: \"kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:19.332458 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:19.332427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:19.334735 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:19.334707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") pod \"maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:19.533291 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:19.533256    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:19.683988 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:19.683958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:19.684405 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:41:19.684346    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5620aa55_d0e5_4fc1_b087_7fec29e81708.slice/crio-5c76dce228ea4443c28b3e48cbdc7550bc69631e9f2b14ea767ece6a335152ad WatchSource:0}: Error finding container 5c76dce228ea4443c28b3e48cbdc7550bc69631e9f2b14ea767ece6a335152ad: Status 404 returned error can't find the container with id 5c76dce228ea4443c28b3e48cbdc7550bc69631e9f2b14ea767ece6a335152ad
Jul 29 01:41:20.006407 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:20.006330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" event={"ID":"5620aa55-d0e5-4fc1-b087-7fec29e81708","Type":"ContainerStarted","Data":"5c76dce228ea4443c28b3e48cbdc7550bc69631e9f2b14ea767ece6a335152ad"}
Jul 29 01:41:21.013117 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:21.013086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" event={"ID":"5620aa55-d0e5-4fc1-b087-7fec29e81708","Type":"ContainerStarted","Data":"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"}
Jul 29 01:41:21.013506 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:21.013222    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:21.036780 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:21.036731    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" podStartSLOduration=2.6521026 podStartE2EDuration="3.036718033s" totalImagesPullingTime="384.615433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:41:21.035213186 +0000 UTC m=+1735.467095665" watchObservedRunningTime="2026-07-29 01:41:21.036718033 +0000 UTC m=+1735.468600512"
Jul 29 01:41:27.026319 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:27.026291    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:44.134665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.134631    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"]
Jul 29 01:41:44.138632 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.138608    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:44.142011 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.141984    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:41:44.152058 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.152015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"]
Jul 29 01:41:44.248554 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.248520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a3670250-c6xv5\" (UID: \"b6eec0b5-74b0-4903-bdcf-c863571f9b87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:44.349526 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.349494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a3670250-c6xv5\" (UID: \"b6eec0b5-74b0-4903-bdcf-c863571f9b87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:44.350048 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.350028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a3670250-c6xv5\" (UID: \"b6eec0b5-74b0-4903-bdcf-c863571f9b87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:44.457551 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.457487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:44.593801 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:44.593770    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"]
Jul 29 01:41:44.595079 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:41:44.595055    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6eec0b5_74b0_4903_bdcf_c863571f9b87.slice/crio-82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229 WatchSource:0}: Error finding container 82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229: Status 404 returned error can't find the container with id 82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229
Jul 29 01:41:45.165912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:45.165814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5" event={"ID":"b6eec0b5-74b0-4903-bdcf-c863571f9b87","Type":"ContainerStarted","Data":"39fcacee76f199ff38ef4feae081c78677f34d8ef0240e52e9524d7e9d607e43"}
Jul 29 01:41:45.165912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:45.165878    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5" event={"ID":"b6eec0b5-74b0-4903-bdcf-c863571f9b87","Type":"ContainerStarted","Data":"82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229"}
Jul 29 01:41:45.185002 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:45.184955    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5" podStartSLOduration=1.184938021 podStartE2EDuration="1.184938021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:41:45.184532857 +0000 UTC m=+1759.616415365" watchObservedRunningTime="2026-07-29 01:41:45.184938021 +0000 UTC m=+1759.616820504"
Jul 29 01:41:46.172600 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:46.172570    2564 generic.go:356] "Generic (PLEG): container finished" podID="b6eec0b5-74b0-4903-bdcf-c863571f9b87" containerID="39fcacee76f199ff38ef4feae081c78677f34d8ef0240e52e9524d7e9d607e43" exitCode=0
Jul 29 01:41:46.172957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:46.172655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5" event={"ID":"b6eec0b5-74b0-4903-bdcf-c863571f9b87","Type":"ContainerDied","Data":"39fcacee76f199ff38ef4feae081c78677f34d8ef0240e52e9524d7e9d607e43"}
Jul 29 01:41:47.320460 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:47.320437    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:47.477518 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:47.477452    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca\") pod \"b6eec0b5-74b0-4903-bdcf-c863571f9b87\" (UID: \"b6eec0b5-74b0-4903-bdcf-c863571f9b87\") "
Jul 29 01:41:47.477779 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:47.477757    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6eec0b5-74b0-4903-bdcf-c863571f9b87" (UID: "b6eec0b5-74b0-4903-bdcf-c863571f9b87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:41:47.578302 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:47.578275    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6eec0b5-74b0-4903-bdcf-c863571f9b87-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:48.185094 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:48.185062    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"
Jul 29 01:41:48.185275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:48.185091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5" event={"ID":"b6eec0b5-74b0-4903-bdcf-c863571f9b87","Type":"ContainerDied","Data":"82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229"}
Jul 29 01:41:48.185275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:48.185197    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82c740d22598485603bd094295ffe02a350d9b8c969ffb8793360c740edca229"
Jul 29 01:41:49.218205 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.218174    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:49.218684 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.218439    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" podUID="5620aa55-d0e5-4fc1-b087-7fec29e81708" containerName="maas-api" containerID="cri-o://f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5" gracePeriod=30
Jul 29 01:41:49.467833 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.467803    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:49.528602 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.528567    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"]
Jul 29 01:41:49.530973 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.530950    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a3670250-c6xv5"]
Jul 29 01:41:49.593277 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.593078    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgpxj\" (UniqueName: \"kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj\") pod \"5620aa55-d0e5-4fc1-b087-7fec29e81708\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") "
Jul 29 01:41:49.593508 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.593121    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") pod \"5620aa55-d0e5-4fc1-b087-7fec29e81708\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") "
Jul 29 01:41:49.593638 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.593587    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca\") pod \"5620aa55-d0e5-4fc1-b087-7fec29e81708\" (UID: \"5620aa55-d0e5-4fc1-b087-7fec29e81708\") "
Jul 29 01:41:49.594014 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.593982    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5620aa55-d0e5-4fc1-b087-7fec29e81708" (UID: "5620aa55-d0e5-4fc1-b087-7fec29e81708"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:41:49.595413 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.595385    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5620aa55-d0e5-4fc1-b087-7fec29e81708" (UID: "5620aa55-d0e5-4fc1-b087-7fec29e81708"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:41:49.595664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.595638    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj" (OuterVolumeSpecName: "kube-api-access-lgpxj") pod "5620aa55-d0e5-4fc1-b087-7fec29e81708" (UID: "5620aa55-d0e5-4fc1-b087-7fec29e81708"). InnerVolumeSpecName "kube-api-access-lgpxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:41:49.694382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.694347    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5620aa55-d0e5-4fc1-b087-7fec29e81708-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:49.694382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.694377    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgpxj\" (UniqueName: \"kubernetes.io/projected/5620aa55-d0e5-4fc1-b087-7fec29e81708-kube-api-access-lgpxj\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:49.694382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:49.694387    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5620aa55-d0e5-4fc1-b087-7fec29e81708-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:41:50.118223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.118192    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6eec0b5-74b0-4903-bdcf-c863571f9b87" path="/var/lib/kubelet/pods/b6eec0b5-74b0-4903-bdcf-c863571f9b87/volumes"
Jul 29 01:41:50.197553 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.197525    2564 generic.go:356] "Generic (PLEG): container finished" podID="5620aa55-d0e5-4fc1-b087-7fec29e81708" containerID="f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5" exitCode=0
Jul 29 01:41:50.197688 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.197573    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"
Jul 29 01:41:50.197688 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.197610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" event={"ID":"5620aa55-d0e5-4fc1-b087-7fec29e81708","Type":"ContainerDied","Data":"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"}
Jul 29 01:41:50.197688 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.197661    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj" event={"ID":"5620aa55-d0e5-4fc1-b087-7fec29e81708","Type":"ContainerDied","Data":"5c76dce228ea4443c28b3e48cbdc7550bc69631e9f2b14ea767ece6a335152ad"}
Jul 29 01:41:50.197797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.197690    2564 scope.go:117] "RemoveContainer" containerID="f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"
Jul 29 01:41:50.207064 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.207037    2564 scope.go:117] "RemoveContainer" containerID="f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"
Jul 29 01:41:50.207325 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:41:50.207306    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5\": container with ID starting with f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5 not found: ID does not exist" containerID="f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"
Jul 29 01:41:50.207386 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.207334    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5"} err="failed to get container status \"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5\": rpc error: code = NotFound desc = could not find container \"f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5\": container with ID starting with f4799752ef0d763d8beea07ed9e060ac7ab115dfcaae67ae147398b7297e1aa5 not found: ID does not exist"
Jul 29 01:41:50.224359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.224315    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:50.227361 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:50.227341    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3670250-5c86f79b4d-4m8pj"]
Jul 29 01:41:52.117477 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:52.117444    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5620aa55-d0e5-4fc1-b087-7fec29e81708" path="/var/lib/kubelet/pods/5620aa55-d0e5-4fc1-b087-7fec29e81708/volumes"
Jul 29 01:41:53.703350 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703317    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:41:53.703726 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703714    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6eec0b5-74b0-4903-bdcf-c863571f9b87" containerName="revoke-keys"
Jul 29 01:41:53.703777 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703727    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eec0b5-74b0-4903-bdcf-c863571f9b87" containerName="revoke-keys"
Jul 29 01:41:53.703777 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703745    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5620aa55-d0e5-4fc1-b087-7fec29e81708" containerName="maas-api"
Jul 29 01:41:53.703777 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703751    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5620aa55-d0e5-4fc1-b087-7fec29e81708" containerName="maas-api"
Jul 29 01:41:53.703881 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703826    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6eec0b5-74b0-4903-bdcf-c863571f9b87" containerName="revoke-keys"
Jul 29 01:41:53.703881 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.703836    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5620aa55-d0e5-4fc1-b087-7fec29e81708" containerName="maas-api"
Jul 29 01:41:53.707079 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.707060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.712515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.712499    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4cab099e-serving-cert\""
Jul 29 01:41:53.717761 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.717737    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:41:53.831697 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.831665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.831697 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.831699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngj6g\" (UniqueName: \"kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.831919 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.831737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.932932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.932903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.933083 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.933015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.933083 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.933040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngj6g\" (UniqueName: \"kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.933701 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.933683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.935428 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.935401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:53.945087 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:53.945064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngj6g\" (UniqueName: \"kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g\") pod \"maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:54.018208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:54.018134    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:54.163544 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:54.163514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:41:54.165031 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:41:54.164996    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa2bc82_5e6f_4dda_8983_5a1e59080601.slice/crio-890004ada935c4b028b271530db6e099630499ec3470030e762514c8796d484c WatchSource:0}: Error finding container 890004ada935c4b028b271530db6e099630499ec3470030e762514c8796d484c: Status 404 returned error can't find the container with id 890004ada935c4b028b271530db6e099630499ec3470030e762514c8796d484c
Jul 29 01:41:54.222139 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:54.222111    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" event={"ID":"8aa2bc82-5e6f-4dda-8983-5a1e59080601","Type":"ContainerStarted","Data":"890004ada935c4b028b271530db6e099630499ec3470030e762514c8796d484c"}
Jul 29 01:41:55.231324 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:55.231288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" event={"ID":"8aa2bc82-5e6f-4dda-8983-5a1e59080601","Type":"ContainerStarted","Data":"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"}
Jul 29 01:41:55.231698 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:55.231363    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:41:55.254169 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:41:55.254106    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" podStartSLOduration=1.78323767 podStartE2EDuration="2.254087556s" totalImagesPullingTime="470.849886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:41:55.250345407 +0000 UTC m=+1769.682227900" watchObservedRunningTime="2026-07-29 01:41:55.254087556 +0000 UTC m=+1769.685970036"
Jul 29 01:42:01.242977 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:01.242939    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:42:18.418913 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.418831    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"]
Jul 29 01:42:18.422404 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.422386    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:18.425011 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.424992    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:42:18.430648 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.430625    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"]
Jul 29 01:42:18.549281 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.549255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4\" (UID: \"fb2766f3-946f-4ea6-a54e-accc984a8b69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:18.650524 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.650482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4\" (UID: \"fb2766f3-946f-4ea6-a54e-accc984a8b69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:18.651087 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.651067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4\" (UID: \"fb2766f3-946f-4ea6-a54e-accc984a8b69\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:18.735751 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.735695    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:18.868495 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:18.868470    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"]
Jul 29 01:42:18.870973 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:42:18.870945    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2766f3_946f_4ea6_a54e_accc984a8b69.slice/crio-b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd WatchSource:0}: Error finding container b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd: Status 404 returned error can't find the container with id b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd
Jul 29 01:42:19.372628 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:19.372560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4" event={"ID":"fb2766f3-946f-4ea6-a54e-accc984a8b69","Type":"ContainerStarted","Data":"7188bbb6e99d35383d09b969f96f6bc01c202a4c139fb401f1b95d98f3bcf623"}
Jul 29 01:42:19.372628 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:19.372623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4" event={"ID":"fb2766f3-946f-4ea6-a54e-accc984a8b69","Type":"ContainerStarted","Data":"b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd"}
Jul 29 01:42:19.391560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:19.391514    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4" podStartSLOduration=1.391500165 podStartE2EDuration="1.391500165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:42:19.390426699 +0000 UTC m=+1793.822309179" watchObservedRunningTime="2026-07-29 01:42:19.391500165 +0000 UTC m=+1793.823382645"
Jul 29 01:42:20.379868 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:20.379837    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb2766f3-946f-4ea6-a54e-accc984a8b69" containerID="7188bbb6e99d35383d09b969f96f6bc01c202a4c139fb401f1b95d98f3bcf623" exitCode=0
Jul 29 01:42:20.380245 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:20.379927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4" event={"ID":"fb2766f3-946f-4ea6-a54e-accc984a8b69","Type":"ContainerDied","Data":"7188bbb6e99d35383d09b969f96f6bc01c202a4c139fb401f1b95d98f3bcf623"}
Jul 29 01:42:21.524954 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:21.524928    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:21.674208 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:21.674091    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca\") pod \"fb2766f3-946f-4ea6-a54e-accc984a8b69\" (UID: \"fb2766f3-946f-4ea6-a54e-accc984a8b69\") "
Jul 29 01:42:21.674481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:21.674456    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fb2766f3-946f-4ea6-a54e-accc984a8b69" (UID: "fb2766f3-946f-4ea6-a54e-accc984a8b69"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:42:21.774759 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:21.774728    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2766f3-946f-4ea6-a54e-accc984a8b69-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:42:22.394869 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:22.394795    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"
Jul 29 01:42:22.394869 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:22.394839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4" event={"ID":"fb2766f3-946f-4ea6-a54e-accc984a8b69","Type":"ContainerDied","Data":"b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd"}
Jul 29 01:42:22.395068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:22.394887    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6737109d098212234147b47e913f8855282eaf11bee5eec91658f885fe98abd"
Jul 29 01:42:24.425203 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.425167    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:42:24.425683 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.425407    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" podUID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" containerName="maas-api" containerID="cri-o://787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5" gracePeriod=30
Jul 29 01:42:24.626897 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.626864    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"]
Jul 29 01:42:24.630358 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.630331    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-4cab099e-hmsq4"]
Jul 29 01:42:24.679525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.679454    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:42:24.800039 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.800006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls\") pod \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") "
Jul 29 01:42:24.800224 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.800143    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngj6g\" (UniqueName: \"kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g\") pod \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") "
Jul 29 01:42:24.800282 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.800249    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca\") pod \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\" (UID: \"8aa2bc82-5e6f-4dda-8983-5a1e59080601\") "
Jul 29 01:42:24.800615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.800585    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8aa2bc82-5e6f-4dda-8983-5a1e59080601" (UID: "8aa2bc82-5e6f-4dda-8983-5a1e59080601"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:42:24.802135 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.802111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8aa2bc82-5e6f-4dda-8983-5a1e59080601" (UID: "8aa2bc82-5e6f-4dda-8983-5a1e59080601"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:42:24.802275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.802246    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g" (OuterVolumeSpecName: "kube-api-access-ngj6g") pod "8aa2bc82-5e6f-4dda-8983-5a1e59080601" (UID: "8aa2bc82-5e6f-4dda-8983-5a1e59080601"). InnerVolumeSpecName "kube-api-access-ngj6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:42:24.901458 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.901421    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa2bc82-5e6f-4dda-8983-5a1e59080601-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:42:24.901458 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.901453    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa2bc82-5e6f-4dda-8983-5a1e59080601-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:42:24.901649 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:24.901468    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngj6g\" (UniqueName: \"kubernetes.io/projected/8aa2bc82-5e6f-4dda-8983-5a1e59080601-kube-api-access-ngj6g\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:42:25.417280 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.417246    2564 generic.go:356] "Generic (PLEG): container finished" podID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" containerID="787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5" exitCode=0
Jul 29 01:42:25.417447 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.417293    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"
Jul 29 01:42:25.417447 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.417338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" event={"ID":"8aa2bc82-5e6f-4dda-8983-5a1e59080601","Type":"ContainerDied","Data":"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"}
Jul 29 01:42:25.417447 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.417387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5" event={"ID":"8aa2bc82-5e6f-4dda-8983-5a1e59080601","Type":"ContainerDied","Data":"890004ada935c4b028b271530db6e099630499ec3470030e762514c8796d484c"}
Jul 29 01:42:25.417447 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.417411    2564 scope.go:117] "RemoveContainer" containerID="787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"
Jul 29 01:42:25.427914 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.427757    2564 scope.go:117] "RemoveContainer" containerID="787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"
Jul 29 01:42:25.428126 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:42:25.428026    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5\": container with ID starting with 787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5 not found: ID does not exist" containerID="787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"
Jul 29 01:42:25.428126 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.428053    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5"} err="failed to get container status \"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5\": rpc error: code = NotFound desc = could not find container \"787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5\": container with ID starting with 787f8555acc438ed5160c142fd4cb83a2d91db4e28e1b7589d6beb0804544bf5 not found: ID does not exist"
Jul 29 01:42:25.449717 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.449690    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:42:25.453265 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:25.453245    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cab099e-5b5b4dd4cd-pgjx5"]
Jul 29 01:42:26.117197 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:26.117166    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" path="/var/lib/kubelet/pods/8aa2bc82-5e6f-4dda-8983-5a1e59080601/volumes"
Jul 29 01:42:26.117524 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:26.117510    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb2766f3-946f-4ea6-a54e-accc984a8b69" path="/var/lib/kubelet/pods/fb2766f3-946f-4ea6-a54e-accc984a8b69/volumes"
Jul 29 01:42:26.179603 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:26.179580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:42:26.187125 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:42:26.187104    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:44:15.916754 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.916679    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:44:15.917237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917116    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb2766f3-946f-4ea6-a54e-accc984a8b69" containerName="revoke-keys"
Jul 29 01:44:15.917237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917129    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb2766f3-946f-4ea6-a54e-accc984a8b69" containerName="revoke-keys"
Jul 29 01:44:15.917237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917173    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" containerName="maas-api"
Jul 29 01:44:15.917237 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917179    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" containerName="maas-api"
Jul 29 01:44:15.917394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917256    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb2766f3-946f-4ea6-a54e-accc984a8b69" containerName="revoke-keys"
Jul 29 01:44:15.917394 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.917266    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa2bc82-5e6f-4dda-8983-5a1e59080601" containerName="maas-api"
Jul 29 01:44:15.920418 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.920402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:15.927665 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.927640    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bc25d452-serving-cert\""
Jul 29 01:44:15.989287 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:15.989260    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:44:16.065982 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.065948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw9gq\" (UniqueName: \"kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.066211 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.066013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.066211 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.066135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.167003 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.166903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.167204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.167031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.167204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.167096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw9gq\" (UniqueName: \"kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.167766 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.167745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.169575 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.169552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.175431 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.175410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw9gq\" (UniqueName: \"kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq\") pod \"maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.230945 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.230919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:16.369565 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:16.369527    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:44:16.370799 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:44:16.370770    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864ace3f_eae1_41c7_9e68_961ced6b1b07.slice/crio-517af7cdfe6e1f36298f93340c8e2660689619eb5f25bc38fbadea1ea62b40eb WatchSource:0}: Error finding container 517af7cdfe6e1f36298f93340c8e2660689619eb5f25bc38fbadea1ea62b40eb: Status 404 returned error can't find the container with id 517af7cdfe6e1f36298f93340c8e2660689619eb5f25bc38fbadea1ea62b40eb
Jul 29 01:44:17.097035 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:17.096952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" event={"ID":"864ace3f-eae1-41c7-9e68-961ced6b1b07","Type":"ContainerStarted","Data":"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"}
Jul 29 01:44:17.097035 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:17.097016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" event={"ID":"864ace3f-eae1-41c7-9e68-961ced6b1b07","Type":"ContainerStarted","Data":"517af7cdfe6e1f36298f93340c8e2660689619eb5f25bc38fbadea1ea62b40eb"}
Jul 29 01:44:17.097481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:17.097116    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:17.116960 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:17.116915    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" podStartSLOduration=1.676532743 podStartE2EDuration="2.116901629s" totalImagesPullingTime="440.368886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:44:17.113941711 +0000 UTC m=+1911.545824191" watchObservedRunningTime="2026-07-29 01:44:17.116901629 +0000 UTC m=+1911.548784143"
Jul 29 01:44:23.108294 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:23.108261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:47.915645 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:47.915611    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"]
Jul 29 01:44:47.919525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:47.919507    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:47.922458 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:47.922434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:44:47.926496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:47.926471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"]
Jul 29 01:44:48.036491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:48.036461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2\" (UID: \"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:48.137412 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:48.137384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2\" (UID: \"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:48.137967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:48.137950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2\" (UID: \"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:48.232327 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:48.232272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:48.372096 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:48.372065    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"]
Jul 29 01:44:48.372864 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:44:48.372833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61da8b1_2a6f_480f_80e1_2f0f8ad8d833.slice/crio-80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7 WatchSource:0}: Error finding container 80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7: Status 404 returned error can't find the container with id 80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7
Jul 29 01:44:49.287851 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:49.287187    2564 generic.go:356] "Generic (PLEG): container finished" podID="f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" containerID="d3d4e9c5efade4bbca5d5a46dd4d12b0349efb4d4ec96db19717fc90099d6918" exitCode=0
Jul 29 01:44:49.287851 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:49.287407    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2" event={"ID":"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833","Type":"ContainerDied","Data":"d3d4e9c5efade4bbca5d5a46dd4d12b0349efb4d4ec96db19717fc90099d6918"}
Jul 29 01:44:49.287851 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:49.287456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2" event={"ID":"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833","Type":"ContainerStarted","Data":"80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7"}
Jul 29 01:44:50.433518 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:50.433491    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:50.556868 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:50.556804    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca\") pod \"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833\" (UID: \"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833\") "
Jul 29 01:44:50.557134 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:50.557111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" (UID: "f61da8b1-2a6f-480f-80e1-2f0f8ad8d833"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:44:50.657605 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:50.657579    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.300848 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:51.300819    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"
Jul 29 01:44:51.300848 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:51.300840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2" event={"ID":"f61da8b1-2a6f-480f-80e1-2f0f8ad8d833","Type":"ContainerDied","Data":"80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7"}
Jul 29 01:44:51.301062 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:51.300882    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80075919c5304d65081dfa2b425fa3f4a54ce9c640e08e6ec520865d1df65ec7"
Jul 29 01:44:58.450477 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.450435    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:44:58.450940 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.450664    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" podUID="864ace3f-eae1-41c7-9e68-961ced6b1b07" containerName="maas-api" containerID="cri-o://6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1" gracePeriod=30
Jul 29 01:44:58.645114 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.645084    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"]
Jul 29 01:44:58.647932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.647909    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc25d452-n8nf2"]
Jul 29 01:44:58.715064 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.715006    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:58.821083 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.821050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls\") pod \"864ace3f-eae1-41c7-9e68-961ced6b1b07\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") "
Jul 29 01:44:58.821238 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.821127    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca\") pod \"864ace3f-eae1-41c7-9e68-961ced6b1b07\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") "
Jul 29 01:44:58.821238 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.821222    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw9gq\" (UniqueName: \"kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq\") pod \"864ace3f-eae1-41c7-9e68-961ced6b1b07\" (UID: \"864ace3f-eae1-41c7-9e68-961ced6b1b07\") "
Jul 29 01:44:58.821533 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.821501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "864ace3f-eae1-41c7-9e68-961ced6b1b07" (UID: "864ace3f-eae1-41c7-9e68-961ced6b1b07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:44:58.823243 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.823210    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "864ace3f-eae1-41c7-9e68-961ced6b1b07" (UID: "864ace3f-eae1-41c7-9e68-961ced6b1b07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:44:58.823378 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.823358    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq" (OuterVolumeSpecName: "kube-api-access-vw9gq") pod "864ace3f-eae1-41c7-9e68-961ced6b1b07" (UID: "864ace3f-eae1-41c7-9e68-961ced6b1b07"). InnerVolumeSpecName "kube-api-access-vw9gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:44:58.922431 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.922409    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/864ace3f-eae1-41c7-9e68-961ced6b1b07-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:44:58.922431 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.922432    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw9gq\" (UniqueName: \"kubernetes.io/projected/864ace3f-eae1-41c7-9e68-961ced6b1b07-kube-api-access-vw9gq\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:44:58.922572 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:58.922441    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/864ace3f-eae1-41c7-9e68-961ced6b1b07-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:44:59.349661 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.349624    2564 generic.go:356] "Generic (PLEG): container finished" podID="864ace3f-eae1-41c7-9e68-961ced6b1b07" containerID="6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1" exitCode=0
Jul 29 01:44:59.349829 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.349669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" event={"ID":"864ace3f-eae1-41c7-9e68-961ced6b1b07","Type":"ContainerDied","Data":"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"}
Jul 29 01:44:59.349829 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.349714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr" event={"ID":"864ace3f-eae1-41c7-9e68-961ced6b1b07","Type":"ContainerDied","Data":"517af7cdfe6e1f36298f93340c8e2660689619eb5f25bc38fbadea1ea62b40eb"}
Jul 29 01:44:59.349829 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.349675    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"
Jul 29 01:44:59.349829 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.349737    2564 scope.go:117] "RemoveContainer" containerID="6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"
Jul 29 01:44:59.360278 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.360261    2564 scope.go:117] "RemoveContainer" containerID="6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"
Jul 29 01:44:59.360518 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:44:59.360498    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1\": container with ID starting with 6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1 not found: ID does not exist" containerID="6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"
Jul 29 01:44:59.360569 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.360526    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1"} err="failed to get container status \"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1\": rpc error: code = NotFound desc = could not find container \"6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1\": container with ID starting with 6a4c99b10cdb2e4d1afc0f8028172901c3a124f049161fb2ba6500a7c22ccec1 not found: ID does not exist"
Jul 29 01:44:59.373077 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.373053    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:44:59.376758 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:44:59.376736    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc25d452-6664848fb6-bhnbr"]
Jul 29 01:45:00.118570 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.118528    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="864ace3f-eae1-41c7-9e68-961ced6b1b07" path="/var/lib/kubelet/pods/864ace3f-eae1-41c7-9e68-961ced6b1b07/volumes"
Jul 29 01:45:00.118963 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.118921    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" path="/var/lib/kubelet/pods/f61da8b1-2a6f-480f-80e1-2f0f8ad8d833/volumes"
Jul 29 01:45:00.132501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132473    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"]
Jul 29 01:45:00.132869 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132857    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="864ace3f-eae1-41c7-9e68-961ced6b1b07" containerName="maas-api"
Jul 29 01:45:00.132912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132870    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="864ace3f-eae1-41c7-9e68-961ced6b1b07" containerName="maas-api"
Jul 29 01:45:00.132912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132882    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" containerName="revoke-keys"
Jul 29 01:45:00.132912 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132887    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" containerName="revoke-keys"
Jul 29 01:45:00.133009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132952    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61da8b1-2a6f-480f-80e1-2f0f8ad8d833" containerName="revoke-keys"
Jul 29 01:45:00.133009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.132962    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="864ace3f-eae1-41c7-9e68-961ced6b1b07" containerName="maas-api"
Jul 29 01:45:00.137273 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.137258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"
Jul 29 01:45:00.140081 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.140064    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:45:00.144510 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.144490    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"]
Jul 29 01:45:00.148948 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.148931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"
Jul 29 01:45:00.291882 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.291849    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"]
Jul 29 01:45:00.294043 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:45:00.294015    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f45283_01a4_4967_8d41_19609a891dcf.slice/crio-91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43 WatchSource:0}: Error finding container 91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43: Status 404 returned error can't find the container with id 91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43
Jul 29 01:45:00.356018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:00.355985    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx" event={"ID":"b3f45283-01a4-4967-8d41-19609a891dcf","Type":"ContainerStarted","Data":"91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43"}
Jul 29 01:45:01.364514 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:01.364428    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3f45283-01a4-4967-8d41-19609a891dcf" containerID="f0548b70a893cf51c8eb5c21441522f86312e25b7356090557d28d5093804149" exitCode=0
Jul 29 01:45:01.364917 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:01.364513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx" event={"ID":"b3f45283-01a4-4967-8d41-19609a891dcf","Type":"ContainerDied","Data":"f0548b70a893cf51c8eb5c21441522f86312e25b7356090557d28d5093804149"}
Jul 29 01:45:02.503252 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:02.503226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"
Jul 29 01:45:03.376899 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:03.376871    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx"
Jul 29 01:45:03.377082 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:03.376921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754825-p92qx" event={"ID":"b3f45283-01a4-4967-8d41-19609a891dcf","Type":"ContainerDied","Data":"91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43"}
Jul 29 01:45:03.377082 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:03.376974    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91945032a5e57a4b875681e65143c6e4014168aa392f20886cabd3a0fe774c43"
Jul 29 01:45:45.312897 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.312863    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:45:45.313413 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.313394    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f45283-01a4-4967-8d41-19609a891dcf" containerName="cleanup"
Jul 29 01:45:45.313459 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.313415    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f45283-01a4-4967-8d41-19609a891dcf" containerName="cleanup"
Jul 29 01:45:45.313529 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.313519    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f45283-01a4-4967-8d41-19609a891dcf" containerName="cleanup"
Jul 29 01:45:45.316885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.316865    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.319454 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.319431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e744ec49-serving-cert\""
Jul 29 01:45:45.330724 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.330703    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:45:45.441676 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.441635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2fj9\" (UniqueName: \"kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.441863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.441697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.441863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.441796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.543373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.543337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.543540 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.543414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.543540 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.543481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2fj9\" (UniqueName: \"kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.543620 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:45:45.543582    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-serving-cert: secret "maas-api-e2e-mt-e744ec49-serving-cert" not found
Jul 29 01:45:45.543693 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:45:45.543680    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls podName:e8f3af47-99cc-476a-9605-2340a761aebc nodeName:}" failed. No retries permitted until 2026-07-29 01:45:46.043656347 +0000 UTC m=+2000.475538805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls") pod "maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" (UID: "e8f3af47-99cc-476a-9605-2340a761aebc") : secret "maas-api-e2e-mt-e744ec49-serving-cert" not found
Jul 29 01:45:45.544107 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.544089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:45.552610 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:45.552582    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2fj9\" (UniqueName: \"kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:46.047519 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.047484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:46.049957 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.049926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") pod \"maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:46.227797 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.227757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:46.400670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.400642    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:45:46.402872 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:45:46.402844    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f3af47_99cc_476a_9605_2340a761aebc.slice/crio-2203a246433fa3fbe4657432431cae34cadd8c622455c2e8d81a88c26d083469 WatchSource:0}: Error finding container 2203a246433fa3fbe4657432431cae34cadd8c622455c2e8d81a88c26d083469: Status 404 returned error can't find the container with id 2203a246433fa3fbe4657432431cae34cadd8c622455c2e8d81a88c26d083469
Jul 29 01:45:46.404348 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.404328    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:45:46.653662 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:46.653568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" event={"ID":"e8f3af47-99cc-476a-9605-2340a761aebc","Type":"ContainerStarted","Data":"2203a246433fa3fbe4657432431cae34cadd8c622455c2e8d81a88c26d083469"}
Jul 29 01:45:47.661643 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:47.661604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" event={"ID":"e8f3af47-99cc-476a-9605-2340a761aebc","Type":"ContainerStarted","Data":"e183b8688c324f2697170b366806e1920dfcc4db383d0d694ce4b69fd15e6f31"}
Jul 29 01:45:47.662055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:47.661671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:45:47.679744 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:47.679688    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" podStartSLOduration=2.089381383 podStartE2EDuration="2.679669778s" totalImagesPullingTime="590.288395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:45:47.677852105 +0000 UTC m=+2002.109734584" watchObservedRunningTime="2026-07-29 01:45:47.679669778 +0000 UTC m=+2002.111552259"
Jul 29 01:45:53.674002 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:45:53.673969    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:46:44.348943 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.348903    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"]
Jul 29 01:46:44.352640 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.352615    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:44.355324 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.355302    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:46:44.362933 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.362908    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"]
Jul 29 01:46:44.456044 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.456018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk\" (UID: \"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:44.557283 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.557182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk\" (UID: \"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:44.557737 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.557717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk\" (UID: \"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:44.665044 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.664959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:44.804030 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:44.804003    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"]
Jul 29 01:46:44.807094 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:46:44.807064    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18bf4d3_817b_4a3f_920f_8fe1fdff25b6.slice/crio-76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084 WatchSource:0}: Error finding container 76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084: Status 404 returned error can't find the container with id 76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084
Jul 29 01:46:45.012164 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:45.012057    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk" event={"ID":"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6","Type":"ContainerStarted","Data":"76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084"}
Jul 29 01:46:46.019585 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:46.019499    2564 generic.go:356] "Generic (PLEG): container finished" podID="b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" containerID="b9a3ba620174d6740a2ca4c430edfe2d0580359cb706017ff866abf703e841b5" exitCode=0
Jul 29 01:46:46.019918 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:46.019584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk" event={"ID":"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6","Type":"ContainerDied","Data":"b9a3ba620174d6740a2ca4c430edfe2d0580359cb706017ff866abf703e841b5"}
Jul 29 01:46:47.171044 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:47.171017    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:47.284395 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:47.284328    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca\") pod \"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6\" (UID: \"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6\") "
Jul 29 01:46:47.284652 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:47.284631    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" (UID: "b18bf4d3-817b-4a3f-920f-8fe1fdff25b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:46:47.386004 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:47.385972    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:46:48.038731 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:48.038694    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"
Jul 29 01:46:48.038910 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:48.038701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk" event={"ID":"b18bf4d3-817b-4a3f-920f-8fe1fdff25b6","Type":"ContainerDied","Data":"76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084"}
Jul 29 01:46:48.038910 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:48.038855    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76fc2a4d033cd7fabfc9f75ec92ecf0f58aa7cd7a26553dbb2c42b9d7de8d084"
Jul 29 01:46:59.894964 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:59.894925    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:46:59.895396 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:46:59.895221    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" podUID="e8f3af47-99cc-476a-9605-2340a761aebc" containerName="maas-api" containerID="cri-o://e183b8688c324f2697170b366806e1920dfcc4db383d0d694ce4b69fd15e6f31" gracePeriod=30
Jul 29 01:47:00.104763 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.104728    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"]
Jul 29 01:47:00.107215 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.107187    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e744ec49-qk4zk"]
Jul 29 01:47:00.115486 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.115287    2564 generic.go:356] "Generic (PLEG): container finished" podID="e8f3af47-99cc-476a-9605-2340a761aebc" containerID="e183b8688c324f2697170b366806e1920dfcc4db383d0d694ce4b69fd15e6f31" exitCode=0
Jul 29 01:47:00.120997 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.120969    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" path="/var/lib/kubelet/pods/b18bf4d3-817b-4a3f-920f-8fe1fdff25b6/volumes"
Jul 29 01:47:00.121297 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.121277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" event={"ID":"e8f3af47-99cc-476a-9605-2340a761aebc","Type":"ContainerDied","Data":"e183b8688c324f2697170b366806e1920dfcc4db383d0d694ce4b69fd15e6f31"}
Jul 29 01:47:00.166328 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.166293    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:47:00.294798 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.294768    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2fj9\" (UniqueName: \"kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9\") pod \"e8f3af47-99cc-476a-9605-2340a761aebc\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") "
Jul 29 01:47:00.294947 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.294846    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") pod \"e8f3af47-99cc-476a-9605-2340a761aebc\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") "
Jul 29 01:47:00.294947 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.294898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca\") pod \"e8f3af47-99cc-476a-9605-2340a761aebc\" (UID: \"e8f3af47-99cc-476a-9605-2340a761aebc\") "
Jul 29 01:47:00.295301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.295277    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8f3af47-99cc-476a-9605-2340a761aebc" (UID: "e8f3af47-99cc-476a-9605-2340a761aebc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:47:00.296863 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.296836    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9" (OuterVolumeSpecName: "kube-api-access-q2fj9") pod "e8f3af47-99cc-476a-9605-2340a761aebc" (UID: "e8f3af47-99cc-476a-9605-2340a761aebc"). InnerVolumeSpecName "kube-api-access-q2fj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:47:00.296973 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.296952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8f3af47-99cc-476a-9605-2340a761aebc" (UID: "e8f3af47-99cc-476a-9605-2340a761aebc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:47:00.395692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.395662    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2fj9\" (UniqueName: \"kubernetes.io/projected/e8f3af47-99cc-476a-9605-2340a761aebc-kube-api-access-q2fj9\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:47:00.395692 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.395689    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8f3af47-99cc-476a-9605-2340a761aebc-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:47:00.395815 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:00.395699    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8f3af47-99cc-476a-9605-2340a761aebc-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:47:01.123313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:01.123284    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"
Jul 29 01:47:01.123799 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:01.123329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt" event={"ID":"e8f3af47-99cc-476a-9605-2340a761aebc","Type":"ContainerDied","Data":"2203a246433fa3fbe4657432431cae34cadd8c622455c2e8d81a88c26d083469"}
Jul 29 01:47:01.123799 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:01.123379    2564 scope.go:117] "RemoveContainer" containerID="e183b8688c324f2697170b366806e1920dfcc4db383d0d694ce4b69fd15e6f31"
Jul 29 01:47:01.146078 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:01.146056    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:47:01.150016 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:01.149992    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e744ec49-6854558cbd-vzjnt"]
Jul 29 01:47:02.117577 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:02.117544    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8f3af47-99cc-476a-9605-2340a761aebc" path="/var/lib/kubelet/pods/e8f3af47-99cc-476a-9605-2340a761aebc/volumes"
Jul 29 01:47:26.218359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:26.218332    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:47:26.229009 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:26.228989    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:47:26.344251 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:26.344225    2564 scope.go:117] "RemoveContainer" containerID="86b91efc5bd5638e862a3b865b43554d9e389da03a6e7823baa95317aa5bc6e3"
Jul 29 01:47:26.352558 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:47:26.352541    2564 scope.go:117] "RemoveContainer" containerID="dece9e98cb74d27a96e78ad9a09e761198cdca295b90c8cb363c581cd3a930fc"
Jul 29 01:48:26.381926 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:26.381890    2564 scope.go:117] "RemoveContainer" containerID="7188bbb6e99d35383d09b969f96f6bc01c202a4c139fb401f1b95d98f3bcf623"
Jul 29 01:48:26.391202 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:26.391179    2564 scope.go:117] "RemoveContainer" containerID="39fcacee76f199ff38ef4feae081c78677f34d8ef0240e52e9524d7e9d607e43"
Jul 29 01:48:50.072014 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.071977    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:48:50.072689 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072667    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8f3af47-99cc-476a-9605-2340a761aebc" containerName="maas-api"
Jul 29 01:48:50.072767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072692    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f3af47-99cc-476a-9605-2340a761aebc" containerName="maas-api"
Jul 29 01:48:50.072767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072707    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" containerName="revoke-keys"
Jul 29 01:48:50.072767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072718    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" containerName="revoke-keys"
Jul 29 01:48:50.072932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072811    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8f3af47-99cc-476a-9605-2340a761aebc" containerName="maas-api"
Jul 29 01:48:50.072932 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.072824    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18bf4d3-817b-4a3f-920f-8fe1fdff25b6" containerName="revoke-keys"
Jul 29 01:48:50.076525 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.076505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.079591 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.079573    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-70bd38-serving-cert\""
Jul 29 01:48:50.089785 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.089758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:48:50.218299 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.218258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.218479 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.218323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbk7h\" (UniqueName: \"kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.218479 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.218370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.319320 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.319282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbk7h\" (UniqueName: \"kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.319516 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.319333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.319516 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.319456    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.320091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.320067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.321882 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.321857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.331204 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.331115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbk7h\" (UniqueName: \"kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h\") pod \"maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.387407 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.387376    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:50.532438 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.532406    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:48:50.533433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:48:50.533407    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc274190e_6be2_49ca_9088_b82c56e3e3e4.slice/crio-80396aa656455969e4aaf7e96fa11677ef34d403c50db445b9cb244e2885843f WatchSource:0}: Error finding container 80396aa656455969e4aaf7e96fa11677ef34d403c50db445b9cb244e2885843f: Status 404 returned error can't find the container with id 80396aa656455969e4aaf7e96fa11677ef34d403c50db445b9cb244e2885843f
Jul 29 01:48:50.793462 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:50.793429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" event={"ID":"c274190e-6be2-49ca-9088-b82c56e3e3e4","Type":"ContainerStarted","Data":"80396aa656455969e4aaf7e96fa11677ef34d403c50db445b9cb244e2885843f"}
Jul 29 01:48:52.806677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:52.806635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" event={"ID":"c274190e-6be2-49ca-9088-b82c56e3e3e4","Type":"ContainerStarted","Data":"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"}
Jul 29 01:48:52.807131 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:52.806762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:48:52.826480 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:52.826437    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" podStartSLOduration=1.2383695989999999 podStartE2EDuration="2.82642149s" totalImagesPullingTime="1.588051891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:48:52.824302393 +0000 UTC m=+2187.256184875" watchObservedRunningTime="2026-07-29 01:48:52.82642149 +0000 UTC m=+2187.258303996"
Jul 29 01:48:58.819093 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:48:58.819063    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:49:18.211868 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.211833    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:49:18.215767 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.215745    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.219079 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.219058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-eb1fe1-serving-cert\""
Jul 29 01:49:18.229468 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.229443    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:49:18.262909 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.262884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.263041 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.262924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdkmx\" (UniqueName: \"kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.263041 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.262966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.363509 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.363470    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.363677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.363523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdkmx\" (UniqueName: \"kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.363677 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.363575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.363749 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:49:18.363709    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-serving-cert: secret "maas-api-e2e-inf-b-eb1fe1-serving-cert" not found
Jul 29 01:49:18.363795 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:49:18.363784    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls podName:65c0bfe1-2a43-4f51-aafc-d315f72d81b6 nodeName:}" failed. No retries permitted until 2026-07-29 01:49:18.863762918 +0000 UTC m=+2213.295645376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls") pod "maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" (UID: "65c0bfe1-2a43-4f51-aafc-d315f72d81b6") : secret "maas-api-e2e-inf-b-eb1fe1-serving-cert" not found
Jul 29 01:49:18.364356 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.364334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.372939 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.372911    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdkmx\" (UniqueName: \"kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.869391 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.869356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:18.871843 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:18.871822    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") pod \"maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:19.128740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:19.128646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:19.273784 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:19.273743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:49:19.274897 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:49:19.274861    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c0bfe1_2a43_4f51_aafc_d315f72d81b6.slice/crio-4c8e4ba3a51e7ec33e8bc3ef3de2545e347db19f9e1e7c5523dd48b956ae0a70 WatchSource:0}: Error finding container 4c8e4ba3a51e7ec33e8bc3ef3de2545e347db19f9e1e7c5523dd48b956ae0a70: Status 404 returned error can't find the container with id 4c8e4ba3a51e7ec33e8bc3ef3de2545e347db19f9e1e7c5523dd48b956ae0a70
Jul 29 01:49:19.969748 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:19.969643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" event={"ID":"65c0bfe1-2a43-4f51-aafc-d315f72d81b6","Type":"ContainerStarted","Data":"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"}
Jul 29 01:49:19.969748 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:19.969711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" event={"ID":"65c0bfe1-2a43-4f51-aafc-d315f72d81b6","Type":"ContainerStarted","Data":"4c8e4ba3a51e7ec33e8bc3ef3de2545e347db19f9e1e7c5523dd48b956ae0a70"}
Jul 29 01:49:19.969748 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:19.969741    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:49:20.013076 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:20.013025    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" podStartSLOduration=1.6869480540000001 podStartE2EDuration="2.013008182s" totalImagesPullingTime="326.060128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:49:20.005935791 +0000 UTC m=+2214.437818273" watchObservedRunningTime="2026-07-29 01:49:20.013008182 +0000 UTC m=+2214.444890663"
Jul 29 01:49:25.990081 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:49:25.990040    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:51:14.010275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.010238    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"]
Jul 29 01:51:14.013948 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.013926    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:14.017529 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.017508    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:51:14.024358 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.024337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"]
Jul 29 01:51:14.060518 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.060495    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb\" (UID: \"1750922a-90ad-4339-9ae2-2eabf84c2c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:14.161227 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.161198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb\" (UID: \"1750922a-90ad-4339-9ae2-2eabf84c2c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:14.161788 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.161768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb\" (UID: \"1750922a-90ad-4339-9ae2-2eabf84c2c49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:14.326196 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.326143    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:14.469612 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.469583    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"]
Jul 29 01:51:14.470871 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:51:14.470837    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1750922a_90ad_4339_9ae2_2eabf84c2c49.slice/crio-53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681 WatchSource:0}: Error finding container 53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681: Status 404 returned error can't find the container with id 53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681
Jul 29 01:51:14.676331 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:14.676260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb" event={"ID":"1750922a-90ad-4339-9ae2-2eabf84c2c49","Type":"ContainerStarted","Data":"53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681"}
Jul 29 01:51:15.683668 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:15.683630    2564 generic.go:356] "Generic (PLEG): container finished" podID="1750922a-90ad-4339-9ae2-2eabf84c2c49" containerID="c5bf1431d6ee06085108452e31e3c54c9144e77bad00c951cc79bcd1085ea4a2" exitCode=0
Jul 29 01:51:15.684033 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:15.683685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb" event={"ID":"1750922a-90ad-4339-9ae2-2eabf84c2c49","Type":"ContainerDied","Data":"c5bf1431d6ee06085108452e31e3c54c9144e77bad00c951cc79bcd1085ea4a2"}
Jul 29 01:51:16.829499 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:16.829469    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:16.885164 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:16.885122    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca\") pod \"1750922a-90ad-4339-9ae2-2eabf84c2c49\" (UID: \"1750922a-90ad-4339-9ae2-2eabf84c2c49\") "
Jul 29 01:51:16.885440 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:16.885418    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1750922a-90ad-4339-9ae2-2eabf84c2c49" (UID: "1750922a-90ad-4339-9ae2-2eabf84c2c49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:16.885529 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:16.885516    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1750922a-90ad-4339-9ae2-2eabf84c2c49-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:17.700128 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:17.700055    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"
Jul 29 01:51:17.700128 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:17.700084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb" event={"ID":"1750922a-90ad-4339-9ae2-2eabf84c2c49","Type":"ContainerDied","Data":"53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681"}
Jul 29 01:51:17.700331 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:17.700133    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53a8702263c0f0aa4ec649b989b4cff86dd58aaf3a3e48b731afece86ecec681"
Jul 29 01:51:24.464043 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.464007    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:51:24.464521 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.464344    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" podUID="c274190e-6be2-49ca-9088-b82c56e3e3e4" containerName="maas-api" containerID="cri-o://407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41" gracePeriod=30
Jul 29 01:51:24.707365 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.707335    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"]
Jul 29 01:51:24.711310 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.711283    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-70bd38-tx2kb"]
Jul 29 01:51:24.717349 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.717291    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:51:24.745735 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.745705    2564 generic.go:356] "Generic (PLEG): container finished" podID="c274190e-6be2-49ca-9088-b82c56e3e3e4" containerID="407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41" exitCode=0
Jul 29 01:51:24.745885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.745759    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"
Jul 29 01:51:24.745885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.745779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" event={"ID":"c274190e-6be2-49ca-9088-b82c56e3e3e4","Type":"ContainerDied","Data":"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"}
Jul 29 01:51:24.745885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.745840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f" event={"ID":"c274190e-6be2-49ca-9088-b82c56e3e3e4","Type":"ContainerDied","Data":"80396aa656455969e4aaf7e96fa11677ef34d403c50db445b9cb244e2885843f"}
Jul 29 01:51:24.745885 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.745873    2564 scope.go:117] "RemoveContainer" containerID="407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"
Jul 29 01:51:24.755371 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.755355    2564 scope.go:117] "RemoveContainer" containerID="407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"
Jul 29 01:51:24.755617 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:51:24.755599    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41\": container with ID starting with 407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41 not found: ID does not exist" containerID="407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"
Jul 29 01:51:24.755668 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.755625    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41"} err="failed to get container status \"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41\": rpc error: code = NotFound desc = could not find container \"407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41\": container with ID starting with 407058ea902c55b3678073166dbefb60ff778b92f9fc298cfc62c0857f31ee41 not found: ID does not exist"
Jul 29 01:51:24.859550 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.859528    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbk7h\" (UniqueName: \"kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h\") pod \"c274190e-6be2-49ca-9088-b82c56e3e3e4\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") "
Jul 29 01:51:24.859650 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.859626    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca\") pod \"c274190e-6be2-49ca-9088-b82c56e3e3e4\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") "
Jul 29 01:51:24.859711 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.859684    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls\") pod \"c274190e-6be2-49ca-9088-b82c56e3e3e4\" (UID: \"c274190e-6be2-49ca-9088-b82c56e3e3e4\") "
Jul 29 01:51:24.860068 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.860023    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c274190e-6be2-49ca-9088-b82c56e3e3e4" (UID: "c274190e-6be2-49ca-9088-b82c56e3e3e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:24.861720 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.861696    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h" (OuterVolumeSpecName: "kube-api-access-jbk7h") pod "c274190e-6be2-49ca-9088-b82c56e3e3e4" (UID: "c274190e-6be2-49ca-9088-b82c56e3e3e4"). InnerVolumeSpecName "kube-api-access-jbk7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:51:24.861873 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.861852    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c274190e-6be2-49ca-9088-b82c56e3e3e4" (UID: "c274190e-6be2-49ca-9088-b82c56e3e3e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:51:24.960617 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.960582    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c274190e-6be2-49ca-9088-b82c56e3e3e4-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:24.960617 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.960615    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c274190e-6be2-49ca-9088-b82c56e3e3e4-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:24.960617 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:24.960624    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbk7h\" (UniqueName: \"kubernetes.io/projected/c274190e-6be2-49ca-9088-b82c56e3e3e4-kube-api-access-jbk7h\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:25.071457 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:25.071431    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:51:25.074091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:25.074068    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-70bd38-5bd57cdd6c-2hf4f"]
Jul 29 01:51:26.117031 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:26.116997    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1750922a-90ad-4339-9ae2-2eabf84c2c49" path="/var/lib/kubelet/pods/1750922a-90ad-4339-9ae2-2eabf84c2c49/volumes"
Jul 29 01:51:26.117443 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:26.117428    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c274190e-6be2-49ca-9088-b82c56e3e3e4" path="/var/lib/kubelet/pods/c274190e-6be2-49ca-9088-b82c56e3e3e4/volumes"
Jul 29 01:51:26.434375 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:26.434299    2564 scope.go:117] "RemoveContainer" containerID="d3d4e9c5efade4bbca5d5a46dd4d12b0349efb4d4ec96db19717fc90099d6918"
Jul 29 01:51:31.787338 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787294    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"]
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787697    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c274190e-6be2-49ca-9088-b82c56e3e3e4" containerName="maas-api"
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787709    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c274190e-6be2-49ca-9088-b82c56e3e3e4" containerName="maas-api"
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787729    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1750922a-90ad-4339-9ae2-2eabf84c2c49" containerName="revoke-keys"
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787736    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1750922a-90ad-4339-9ae2-2eabf84c2c49" containerName="revoke-keys"
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787803    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1750922a-90ad-4339-9ae2-2eabf84c2c49" containerName="revoke-keys"
Jul 29 01:51:31.787891 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.787812    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c274190e-6be2-49ca-9088-b82c56e3e3e4" containerName="maas-api"
Jul 29 01:51:31.790904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.790885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:31.793729 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.793712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:51:31.797242 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.797220    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"]
Jul 29 01:51:31.919593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:31.919568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz\" (UID: \"c3644d0c-9001-48e4-a575-b2da3f7dfc5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:32.020740 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.020702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz\" (UID: \"c3644d0c-9001-48e4-a575-b2da3f7dfc5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:32.021364 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.021343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz\" (UID: \"c3644d0c-9001-48e4-a575-b2da3f7dfc5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:32.102772 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.102711    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:32.236551 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.236523    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"]
Jul 29 01:51:32.238433 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:51:32.238406    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3644d0c_9001_48e4_a575_b2da3f7dfc5a.slice/crio-46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e WatchSource:0}: Error finding container 46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e: Status 404 returned error can't find the container with id 46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e
Jul 29 01:51:32.796604 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.796573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz" event={"ID":"c3644d0c-9001-48e4-a575-b2da3f7dfc5a","Type":"ContainerStarted","Data":"35ff70beef179a1b4da7108309602ce22078852c7d4940d98e77ea47493eb7aa"}
Jul 29 01:51:32.796959 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.796624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz" event={"ID":"c3644d0c-9001-48e4-a575-b2da3f7dfc5a","Type":"ContainerStarted","Data":"46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e"}
Jul 29 01:51:32.813251 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:32.813206    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz" podStartSLOduration=1.813191349 podStartE2EDuration="1.813191349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:51:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:51:32.812421649 +0000 UTC m=+2347.244304128" watchObservedRunningTime="2026-07-29 01:51:32.813191349 +0000 UTC m=+2347.245073829"
Jul 29 01:51:33.804012 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:33.803979    2564 generic.go:356] "Generic (PLEG): container finished" podID="c3644d0c-9001-48e4-a575-b2da3f7dfc5a" containerID="35ff70beef179a1b4da7108309602ce22078852c7d4940d98e77ea47493eb7aa" exitCode=0
Jul 29 01:51:33.804428 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:33.804054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz" event={"ID":"c3644d0c-9001-48e4-a575-b2da3f7dfc5a","Type":"ContainerDied","Data":"35ff70beef179a1b4da7108309602ce22078852c7d4940d98e77ea47493eb7aa"}
Jul 29 01:51:34.947421 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:34.947399    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:35.051217 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.051187    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca\") pod \"c3644d0c-9001-48e4-a575-b2da3f7dfc5a\" (UID: \"c3644d0c-9001-48e4-a575-b2da3f7dfc5a\") "
Jul 29 01:51:35.051560 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.051535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3644d0c-9001-48e4-a575-b2da3f7dfc5a" (UID: "c3644d0c-9001-48e4-a575-b2da3f7dfc5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:35.152269 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.152196    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3644d0c-9001-48e4-a575-b2da3f7dfc5a-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:35.816733 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.816700    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"
Jul 29 01:51:35.816733 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.816722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz" event={"ID":"c3644d0c-9001-48e4-a575-b2da3f7dfc5a","Type":"ContainerDied","Data":"46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e"}
Jul 29 01:51:35.816947 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:35.816771    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46797f1f45531fe2ca5fd4132fe6a2288b0fcf7c5b122c839b955ee4f2b9f40e"
Jul 29 01:51:37.342085 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.342040    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:51:37.342591 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.342339    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" podUID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" containerName="maas-api" containerID="cri-o://da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922" gracePeriod=30
Jul 29 01:51:37.548018 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.547982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"]
Jul 29 01:51:37.551825 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.551793    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-eb1fe1-t4jgz"]
Jul 29 01:51:37.607408 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.607355    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:51:37.775199 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.775143    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") pod \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") "
Jul 29 01:51:37.775359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.775279    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdkmx\" (UniqueName: \"kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx\") pod \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") "
Jul 29 01:51:37.775359 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.775301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca\") pod \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\" (UID: \"65c0bfe1-2a43-4f51-aafc-d315f72d81b6\") "
Jul 29 01:51:37.775695 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.775660    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65c0bfe1-2a43-4f51-aafc-d315f72d81b6" (UID: "65c0bfe1-2a43-4f51-aafc-d315f72d81b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:37.777253 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.777221    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65c0bfe1-2a43-4f51-aafc-d315f72d81b6" (UID: "65c0bfe1-2a43-4f51-aafc-d315f72d81b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:51:37.777504 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.777481    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx" (OuterVolumeSpecName: "kube-api-access-hdkmx") pod "65c0bfe1-2a43-4f51-aafc-d315f72d81b6" (UID: "65c0bfe1-2a43-4f51-aafc-d315f72d81b6"). InnerVolumeSpecName "kube-api-access-hdkmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:51:37.829422 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.829395    2564 generic.go:356] "Generic (PLEG): container finished" podID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" containerID="da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922" exitCode=0
Jul 29 01:51:37.829522 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.829442    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"
Jul 29 01:51:37.829522 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.829487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" event={"ID":"65c0bfe1-2a43-4f51-aafc-d315f72d81b6","Type":"ContainerDied","Data":"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"}
Jul 29 01:51:37.829653 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.829559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2" event={"ID":"65c0bfe1-2a43-4f51-aafc-d315f72d81b6","Type":"ContainerDied","Data":"4c8e4ba3a51e7ec33e8bc3ef3de2545e347db19f9e1e7c5523dd48b956ae0a70"}
Jul 29 01:51:37.829653 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.829595    2564 scope.go:117] "RemoveContainer" containerID="da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"
Jul 29 01:51:37.839986 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.839967    2564 scope.go:117] "RemoveContainer" containerID="da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"
Jul 29 01:51:37.840261 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:51:37.840240    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922\": container with ID starting with da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922 not found: ID does not exist" containerID="da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"
Jul 29 01:51:37.840335 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.840273    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922"} err="failed to get container status \"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922\": rpc error: code = NotFound desc = could not find container \"da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922\": container with ID starting with da0ea8341bf2b3474f5e997e4c4883215f658831b48845bbfbd5ce92e48aa922 not found: ID does not exist"
Jul 29 01:51:37.851791 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.851770    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:51:37.855593 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.855563    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-eb1fe1-54b75b89fb-st2f2"]
Jul 29 01:51:37.876615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.876565    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdkmx\" (UniqueName: \"kubernetes.io/projected/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-kube-api-access-hdkmx\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:37.876615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.876594    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:37.876615 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:37.876611    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65c0bfe1-2a43-4f51-aafc-d315f72d81b6-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:51:38.116664 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:38.116636    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" path="/var/lib/kubelet/pods/65c0bfe1-2a43-4f51-aafc-d315f72d81b6/volumes"
Jul 29 01:51:38.116973 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:38.116960    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3644d0c-9001-48e4-a575-b2da3f7dfc5a" path="/var/lib/kubelet/pods/c3644d0c-9001-48e4-a575-b2da3f7dfc5a/volumes"
Jul 29 01:51:47.962127 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962085    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:51:47.962751 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962733    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" containerName="maas-api"
Jul 29 01:51:47.962800 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962754    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" containerName="maas-api"
Jul 29 01:51:47.962800 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962788    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3644d0c-9001-48e4-a575-b2da3f7dfc5a" containerName="revoke-keys"
Jul 29 01:51:47.962800 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962796    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3644d0c-9001-48e4-a575-b2da3f7dfc5a" containerName="revoke-keys"
Jul 29 01:51:47.962902 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962893    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3644d0c-9001-48e4-a575-b2da3f7dfc5a" containerName="revoke-keys"
Jul 29 01:51:47.962940 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.962913    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c0bfe1-2a43-4f51-aafc-d315f72d81b6" containerName="maas-api"
Jul 29 01:51:47.967843 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.967819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:47.970743 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.970719    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6127b1-serving-cert\""
Jul 29 01:51:47.977506 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:47.977484    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:51:48.071793 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.071759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.071967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.071914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzj6v\" (UniqueName: \"kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.071967 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.071960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.173494 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.173452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzj6v\" (UniqueName: \"kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.173494 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.173496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.173718 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.173585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.173799 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:51:48.173768    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-serving-cert: secret "maas-api-e2e-api-a-6127b1-serving-cert" not found
Jul 29 01:51:48.173875 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:51:48.173860    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls podName:01972d54-0d1b-4289-9308-8ee5bb18e312 nodeName:}" failed. No retries permitted until 2026-07-29 01:51:48.673836848 +0000 UTC m=+2363.105719305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls") pod "maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" (UID: "01972d54-0d1b-4289-9308-8ee5bb18e312") : secret "maas-api-e2e-api-a-6127b1-serving-cert" not found
Jul 29 01:51:48.174277 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.174256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.182951 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.182919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzj6v\" (UniqueName: \"kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.677984 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.677947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.680448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.680424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") pod \"maas-api-e2e-api-a-6127b1-64cc598495-zqmnt\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:48.880475 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:48.880415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:49.033928 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.033896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:51:49.035320 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:51:49.035294    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01972d54_0d1b_4289_9308_8ee5bb18e312.slice/crio-43bd99549470defdf6d435bc24a30b9bfb4e5fe6674ee8dee8401aecbcce502f WatchSource:0}: Error finding container 43bd99549470defdf6d435bc24a30b9bfb4e5fe6674ee8dee8401aecbcce502f: Status 404 returned error can't find the container with id 43bd99549470defdf6d435bc24a30b9bfb4e5fe6674ee8dee8401aecbcce502f
Jul 29 01:51:49.036824 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.036802    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:51:49.913904 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.913809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" event={"ID":"01972d54-0d1b-4289-9308-8ee5bb18e312","Type":"ContainerStarted","Data":"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"}
Jul 29 01:51:49.914049 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.913905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" event={"ID":"01972d54-0d1b-4289-9308-8ee5bb18e312","Type":"ContainerStarted","Data":"43bd99549470defdf6d435bc24a30b9bfb4e5fe6674ee8dee8401aecbcce502f"}
Jul 29 01:51:49.914049 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.913933    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:51:49.932133 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:49.932087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" podStartSLOduration=2.329411485 podStartE2EDuration="2.932073731s" totalImagesPullingTime="602.662246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:51:49.931493258 +0000 UTC m=+2364.363375737" watchObservedRunningTime="2026-07-29 01:51:49.932073731 +0000 UTC m=+2364.363956211"
Jul 29 01:51:55.925444 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:51:55.925411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:52:16.164098 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.164058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:52:16.168746 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.168719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.171793 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.171772    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-66baf8-serving-cert\""
Jul 29 01:52:16.187327 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.187302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:52:16.312085 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.312051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.312259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.312115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj7xm\" (UniqueName: \"kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.312259 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.312180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.413515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.413478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj7xm\" (UniqueName: \"kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.413515 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.413517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.413748 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:52:16.413647    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-serving-cert: secret "maas-api-e2e-api-b-66baf8-serving-cert" not found
Jul 29 01:52:16.413748 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:52:16.413705    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls podName:970be4eb-8e5a-4d6f-9430-73eca8d05702 nodeName:}" failed. No retries permitted until 2026-07-29 01:52:16.913689874 +0000 UTC m=+2391.345572332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls") pod "maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" (UID: "970be4eb-8e5a-4d6f-9430-73eca8d05702") : secret "maas-api-e2e-api-b-66baf8-serving-cert" not found
Jul 29 01:52:16.413748 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.413702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.414370 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.414304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.422853 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.422836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj7xm\" (UniqueName: \"kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.919004 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.918969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:16.921384 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:16.921357    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") pod \"maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:17.078820 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:17.078789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:17.222508 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:17.222475    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:52:17.222911 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:52:17.222826    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970be4eb_8e5a_4d6f_9430_73eca8d05702.slice/crio-c7aef2e2899ab59ac2c0711954cb80c31ef80b512a46f193f136756026f31b3b WatchSource:0}: Error finding container c7aef2e2899ab59ac2c0711954cb80c31ef80b512a46f193f136756026f31b3b: Status 404 returned error can't find the container with id c7aef2e2899ab59ac2c0711954cb80c31ef80b512a46f193f136756026f31b3b
Jul 29 01:52:18.107454 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:18.107415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" event={"ID":"970be4eb-8e5a-4d6f-9430-73eca8d05702","Type":"ContainerStarted","Data":"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"}
Jul 29 01:52:18.107659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:18.107472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" event={"ID":"970be4eb-8e5a-4d6f-9430-73eca8d05702","Type":"ContainerStarted","Data":"c7aef2e2899ab59ac2c0711954cb80c31ef80b512a46f193f136756026f31b3b"}
Jul 29 01:52:18.107659 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:18.107530    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:18.126410 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:18.126363    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" podStartSLOduration=1.691563911 podStartE2EDuration="2.126350833s" totalImagesPullingTime="434.786922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:52:18.12362018 +0000 UTC m=+2392.555502663" watchObservedRunningTime="2026-07-29 01:52:18.126350833 +0000 UTC m=+2392.558233362"
Jul 29 01:52:24.119223 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:24.119195    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:52:26.270335 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:26.270293    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:52:26.279057 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:26.279035    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:52:43.826759 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.826723    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"]
Jul 29 01:52:43.830501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.830483    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:43.833105 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.833083    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:52:43.838995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.838972    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"]
Jul 29 01:52:43.851234 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.851208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs\" (UID: \"cd683e16-df87-4517-896b-6703ac2afaec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:43.952579 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.952542    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs\" (UID: \"cd683e16-df87-4517-896b-6703ac2afaec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:43.953414 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:43.953394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs\" (UID: \"cd683e16-df87-4517-896b-6703ac2afaec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:44.143277 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:44.143198    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:44.299193 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:44.299168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"]
Jul 29 01:52:44.301050 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:52:44.301020    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd683e16_df87_4517_896b_6703ac2afaec.slice/crio-07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897 WatchSource:0}: Error finding container 07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897: Status 404 returned error can't find the container with id 07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897
Jul 29 01:52:45.268961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:45.268924    2564 generic.go:356] "Generic (PLEG): container finished" podID="cd683e16-df87-4517-896b-6703ac2afaec" containerID="92f54ce51ff439749d2f46aae78c47f845be63bdd900ec9db7c3feca416056f1" exitCode=0
Jul 29 01:52:45.269382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:45.268987    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs" event={"ID":"cd683e16-df87-4517-896b-6703ac2afaec","Type":"ContainerDied","Data":"92f54ce51ff439749d2f46aae78c47f845be63bdd900ec9db7c3feca416056f1"}
Jul 29 01:52:45.269382 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:45.269025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs" event={"ID":"cd683e16-df87-4517-896b-6703ac2afaec","Type":"ContainerStarted","Data":"07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897"}
Jul 29 01:52:46.415496 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:46.415467    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:46.475010 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:46.474974    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca\") pod \"cd683e16-df87-4517-896b-6703ac2afaec\" (UID: \"cd683e16-df87-4517-896b-6703ac2afaec\") "
Jul 29 01:52:46.475346 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:46.475323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd683e16-df87-4517-896b-6703ac2afaec" (UID: "cd683e16-df87-4517-896b-6703ac2afaec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:46.576313 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:46.576232    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd683e16-df87-4517-896b-6703ac2afaec-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:52:47.283823 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:47.283790    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"
Jul 29 01:52:47.283995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:47.283832    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs" event={"ID":"cd683e16-df87-4517-896b-6703ac2afaec","Type":"ContainerDied","Data":"07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897"}
Jul 29 01:52:47.283995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:47.283876    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07be2c74aec317f58175094eb30247d5797c0e7a91dd21ef4f1650804748a897"
Jul 29 01:52:59.290160 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.290111    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:52:59.291080 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.290422    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" podUID="01972d54-0d1b-4289-9308-8ee5bb18e312" containerName="maas-api" containerID="cri-o://46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447" gracePeriod=30
Jul 29 01:52:59.490686 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.490652    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"]
Jul 29 01:52:59.496390 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.496361    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-6127b1-ws4rs"]
Jul 29 01:52:59.553585 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.553523    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:52:59.580348 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.580322    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzj6v\" (UniqueName: \"kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v\") pod \"01972d54-0d1b-4289-9308-8ee5bb18e312\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") "
Jul 29 01:52:59.580463 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.580381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") pod \"01972d54-0d1b-4289-9308-8ee5bb18e312\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") "
Jul 29 01:52:59.580463 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.580434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca\") pod \"01972d54-0d1b-4289-9308-8ee5bb18e312\" (UID: \"01972d54-0d1b-4289-9308-8ee5bb18e312\") "
Jul 29 01:52:59.580818 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.580792    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01972d54-0d1b-4289-9308-8ee5bb18e312" (UID: "01972d54-0d1b-4289-9308-8ee5bb18e312"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:59.582503 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.582482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v" (OuterVolumeSpecName: "kube-api-access-gzj6v") pod "01972d54-0d1b-4289-9308-8ee5bb18e312" (UID: "01972d54-0d1b-4289-9308-8ee5bb18e312"). InnerVolumeSpecName "kube-api-access-gzj6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:59.582591 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.582517    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01972d54-0d1b-4289-9308-8ee5bb18e312" (UID: "01972d54-0d1b-4289-9308-8ee5bb18e312"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:52:59.681883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.681849    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01972d54-0d1b-4289-9308-8ee5bb18e312-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:52:59.681883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.681878    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01972d54-0d1b-4289-9308-8ee5bb18e312-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:52:59.681883 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:52:59.681887    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzj6v\" (UniqueName: \"kubernetes.io/projected/01972d54-0d1b-4289-9308-8ee5bb18e312-kube-api-access-gzj6v\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:53:00.117289 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.117255    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd683e16-df87-4517-896b-6703ac2afaec" path="/var/lib/kubelet/pods/cd683e16-df87-4517-896b-6703ac2afaec/volumes"
Jul 29 01:53:00.366065 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.366032    2564 generic.go:356] "Generic (PLEG): container finished" podID="01972d54-0d1b-4289-9308-8ee5bb18e312" containerID="46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447" exitCode=0
Jul 29 01:53:00.366563 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.366073    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"
Jul 29 01:53:00.366563 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.366122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" event={"ID":"01972d54-0d1b-4289-9308-8ee5bb18e312","Type":"ContainerDied","Data":"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"}
Jul 29 01:53:00.366563 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.366199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt" event={"ID":"01972d54-0d1b-4289-9308-8ee5bb18e312","Type":"ContainerDied","Data":"43bd99549470defdf6d435bc24a30b9bfb4e5fe6674ee8dee8401aecbcce502f"}
Jul 29 01:53:00.366563 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.366233    2564 scope.go:117] "RemoveContainer" containerID="46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"
Jul 29 01:53:00.378046 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.378025    2564 scope.go:117] "RemoveContainer" containerID="46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"
Jul 29 01:53:00.378338 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:53:00.378312    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447\": container with ID starting with 46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447 not found: ID does not exist" containerID="46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"
Jul 29 01:53:00.378420 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.378346    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447"} err="failed to get container status \"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447\": rpc error: code = NotFound desc = could not find container \"46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447\": container with ID starting with 46a8e0fc2425a747506d3d167231335874a147037f6c3615ed52d270c074a447 not found: ID does not exist"
Jul 29 01:53:00.386561 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.386540    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:53:00.388979 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:00.388960    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6127b1-64cc598495-zqmnt"]
Jul 29 01:53:02.116835 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:02.116802    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01972d54-0d1b-4289-9308-8ee5bb18e312" path="/var/lib/kubelet/pods/01972d54-0d1b-4289-9308-8ee5bb18e312/volumes"
Jul 29 01:53:06.158355 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158325    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"]
Jul 29 01:53:06.158790 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158709    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd683e16-df87-4517-896b-6703ac2afaec" containerName="revoke-keys"
Jul 29 01:53:06.158790 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158719    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd683e16-df87-4517-896b-6703ac2afaec" containerName="revoke-keys"
Jul 29 01:53:06.158790 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158742    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01972d54-0d1b-4289-9308-8ee5bb18e312" containerName="maas-api"
Jul 29 01:53:06.158790 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158747    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="01972d54-0d1b-4289-9308-8ee5bb18e312" containerName="maas-api"
Jul 29 01:53:06.158923 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158809    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="01972d54-0d1b-4289-9308-8ee5bb18e312" containerName="maas-api"
Jul 29 01:53:06.158923 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.158818    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd683e16-df87-4517-896b-6703ac2afaec" containerName="revoke-keys"
Jul 29 01:53:06.162091 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.162075    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:06.164769 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.164751    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 01:53:06.171228 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.171207    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"]
Jul 29 01:53:06.239113 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.239084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6\" (UID: \"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:06.339749 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.339722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6\" (UID: \"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:06.340373 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.340353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6\" (UID: \"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:06.473922 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.473847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:06.608655 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:06.608626    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"]
Jul 29 01:53:06.610264 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:53:06.610238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b1915e_17b6_4b3c_a93f_c6dca2cc43e0.slice/crio-387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760 WatchSource:0}: Error finding container 387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760: Status 404 returned error can't find the container with id 387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760
Jul 29 01:53:07.409980 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:07.409949    2564 generic.go:356] "Generic (PLEG): container finished" podID="42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" containerID="0e0d118c0561a80110061fbd6aeb501be1c2876b758e24f9ff0477144d253d0b" exitCode=0
Jul 29 01:53:07.410300 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:07.409992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6" event={"ID":"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0","Type":"ContainerDied","Data":"0e0d118c0561a80110061fbd6aeb501be1c2876b758e24f9ff0477144d253d0b"}
Jul 29 01:53:07.410300 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:07.410061    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6" event={"ID":"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0","Type":"ContainerStarted","Data":"387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760"}
Jul 29 01:53:08.552049 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:08.552018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:08.661636 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:08.661600    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca\") pod \"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0\" (UID: \"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0\") "
Jul 29 01:53:08.661943 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:08.661922    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" (UID: "42b1915e-17b6-4b3c-a93f-c6dca2cc43e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:53:08.763275 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:08.763210    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:53:09.426481 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:09.426452    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"
Jul 29 01:53:09.426670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:09.426452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6" event={"ID":"42b1915e-17b6-4b3c-a93f-c6dca2cc43e0","Type":"ContainerDied","Data":"387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760"}
Jul 29 01:53:09.426670 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:09.426601    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="387d75ddda0c9a6b400cea447ac2e6484d7c7a21efe648d7bf5015b66a791760"
Jul 29 01:53:21.581055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.579216    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:53:21.581055 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.579454    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" podUID="970be4eb-8e5a-4d6f-9430-73eca8d05702" containerName="maas-api" containerID="cri-o://214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9" gracePeriod=30
Jul 29 01:53:21.783936 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.783899    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"]
Jul 29 01:53:21.788032 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.787997    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-66baf8-dcxk6"]
Jul 29 01:53:21.836089 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.836028    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:53:21.974724 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.974692    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj7xm\" (UniqueName: \"kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm\") pod \"970be4eb-8e5a-4d6f-9430-73eca8d05702\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") "
Jul 29 01:53:21.974925 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.974827    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") pod \"970be4eb-8e5a-4d6f-9430-73eca8d05702\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") "
Jul 29 01:53:21.974925 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.974894    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca\") pod \"970be4eb-8e5a-4d6f-9430-73eca8d05702\" (UID: \"970be4eb-8e5a-4d6f-9430-73eca8d05702\") "
Jul 29 01:53:21.975238 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.975200    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "970be4eb-8e5a-4d6f-9430-73eca8d05702" (UID: "970be4eb-8e5a-4d6f-9430-73eca8d05702"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:53:21.976965 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.976927    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "970be4eb-8e5a-4d6f-9430-73eca8d05702" (UID: "970be4eb-8e5a-4d6f-9430-73eca8d05702"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:53:21.977075 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:21.976990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm" (OuterVolumeSpecName: "kube-api-access-mj7xm") pod "970be4eb-8e5a-4d6f-9430-73eca8d05702" (UID: "970be4eb-8e5a-4d6f-9430-73eca8d05702"). InnerVolumeSpecName "kube-api-access-mj7xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:53:22.076436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.076398    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970be4eb-8e5a-4d6f-9430-73eca8d05702-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:53:22.076436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.076433    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj7xm\" (UniqueName: \"kubernetes.io/projected/970be4eb-8e5a-4d6f-9430-73eca8d05702-kube-api-access-mj7xm\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:53:22.076607 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.076447    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970be4eb-8e5a-4d6f-9430-73eca8d05702-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 01:53:22.117307 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.117241    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" path="/var/lib/kubelet/pods/42b1915e-17b6-4b3c-a93f-c6dca2cc43e0/volumes"
Jul 29 01:53:22.514961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.514880    2564 generic.go:356] "Generic (PLEG): container finished" podID="970be4eb-8e5a-4d6f-9430-73eca8d05702" containerID="214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9" exitCode=0
Jul 29 01:53:22.514961 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.514925    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"
Jul 29 01:53:22.515138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.514970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" event={"ID":"970be4eb-8e5a-4d6f-9430-73eca8d05702","Type":"ContainerDied","Data":"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"}
Jul 29 01:53:22.515138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.515022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc" event={"ID":"970be4eb-8e5a-4d6f-9430-73eca8d05702","Type":"ContainerDied","Data":"c7aef2e2899ab59ac2c0711954cb80c31ef80b512a46f193f136756026f31b3b"}
Jul 29 01:53:22.515138 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.515046    2564 scope.go:117] "RemoveContainer" containerID="214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"
Jul 29 01:53:22.524337 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.524307    2564 scope.go:117] "RemoveContainer" containerID="214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"
Jul 29 01:53:22.524580 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:53:22.524561    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9\": container with ID starting with 214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9 not found: ID does not exist" containerID="214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"
Jul 29 01:53:22.524632 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.524589    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9"} err="failed to get container status \"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9\": rpc error: code = NotFound desc = could not find container \"214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9\": container with ID starting with 214b306cd1b586cfa6589cefe202c26ee9cc180fd6c420126d22df9f8f812cd9 not found: ID does not exist"
Jul 29 01:53:22.535196 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.535172    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:53:22.539301 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:22.539282    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-66baf8-87cf678d4-ccvbc"]
Jul 29 01:53:24.117501 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:24.117468    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970be4eb-8e5a-4d6f-9430-73eca8d05702" path="/var/lib/kubelet/pods/970be4eb-8e5a-4d6f-9430-73eca8d05702/volumes"
Jul 29 01:53:26.477890 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:26.477849    2564 scope.go:117] "RemoveContainer" containerID="b9a3ba620174d6740a2ca4c430edfe2d0580359cb706017ff866abf703e841b5"
Jul 29 01:53:32.356196 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356162    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 01:53:32.356602 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356589    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970be4eb-8e5a-4d6f-9430-73eca8d05702" containerName="maas-api"
Jul 29 01:53:32.356602 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356604    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="970be4eb-8e5a-4d6f-9430-73eca8d05702" containerName="maas-api"
Jul 29 01:53:32.356679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356627    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" containerName="revoke-keys"
Jul 29 01:53:32.356679 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356636    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" containerName="revoke-keys"
Jul 29 01:53:32.356744 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356721    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="970be4eb-8e5a-4d6f-9430-73eca8d05702" containerName="maas-api"
Jul 29 01:53:32.356744 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.356732    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b1915e-17b6-4b3c-a93f-c6dca2cc43e0" containerName="revoke-keys"
Jul 29 01:53:32.359871 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.359853    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.362780 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.362755    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-c798cc-serving-cert\""
Jul 29 01:53:32.372859 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.372838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 01:53:32.464268 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.464212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blmw7\" (UniqueName: \"kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.464433 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.464281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.464433 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.464353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.565294 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.565262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blmw7\" (UniqueName: \"kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.565453 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.565302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.565453 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.565331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.565993 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.565968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.568105 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.568075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.574527 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.574508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blmw7\" (UniqueName: \"kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7\") pod \"maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.673741 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.673669    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:32.820526 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:32.820501    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 01:53:32.821938 ip-10-0-141-49 kubenswrapper[2564]: W0729 01:53:32.821911    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbab41c_bd5c_49d5_9fe8_73c8293543ee.slice/crio-905b1b8c740f42a14bb462176aaf9dbc39b61e9358145b15b54782c01639a134 WatchSource:0}: Error finding container 905b1b8c740f42a14bb462176aaf9dbc39b61e9358145b15b54782c01639a134: Status 404 returned error can't find the container with id 905b1b8c740f42a14bb462176aaf9dbc39b61e9358145b15b54782c01639a134
Jul 29 01:53:33.594604 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:33.594570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" event={"ID":"ffbab41c-bd5c-49d5-9fe8-73c8293543ee","Type":"ContainerStarted","Data":"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"}
Jul 29 01:53:33.594974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:33.594623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" event={"ID":"ffbab41c-bd5c-49d5-9fe8-73c8293543ee","Type":"ContainerStarted","Data":"905b1b8c740f42a14bb462176aaf9dbc39b61e9358145b15b54782c01639a134"}
Jul 29 01:53:33.594974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:33.594651    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:53:33.614917 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:33.614865    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" podStartSLOduration=1.067285903 podStartE2EDuration="1.614850138s" totalImagesPullingTime="547.564235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:53:33.614640121 +0000 UTC m=+2468.046522600" watchObservedRunningTime="2026-07-29 01:53:33.614850138 +0000 UTC m=+2468.046732618"
Jul 29 01:53:39.605856 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:53:39.605826    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 01:54:00.534899 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.534858    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 01:54:00.540491 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.540465    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.543271 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.543250    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0d32f5-serving-cert\""
Jul 29 01:54:00.551448 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.551421    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 01:54:00.604974 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.604947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.605123 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.604994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmqhg\" (UniqueName: \"kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.605231 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.605189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.706100 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.706061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.706299 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.706263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.706388 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.706312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmqhg\" (UniqueName: \"kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.706388 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:54:00.706331    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-serving-cert: secret "maas-api-e2e-shared-b-0d32f5-serving-cert" not found
Jul 29 01:54:00.706492 ip-10-0-141-49 kubenswrapper[2564]: E0729 01:54:00.706405    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls podName:f3b576e7-c153-453e-95cf-547817cb24d2 nodeName:}" failed. No retries permitted until 2026-07-29 01:54:01.206383313 +0000 UTC m=+2495.638265777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls") pod "maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" (UID: "f3b576e7-c153-453e-95cf-547817cb24d2") : secret "maas-api-e2e-shared-b-0d32f5-serving-cert" not found
Jul 29 01:54:00.706840 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.706816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:00.718952 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:00.718926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmqhg\" (UniqueName: \"kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:01.211141 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:01.211100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:01.213661 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:01.213635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") pod \"maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:01.453655 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:01.453620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:01.595661 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:01.595635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 01:54:01.776471 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:01.776437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" event={"ID":"f3b576e7-c153-453e-95cf-547817cb24d2","Type":"ContainerStarted","Data":"bb1ae57ec36e5a3bb6ac7912ebeff92d93b67b457b32502e44d6a5e693d5c927"}
Jul 29 01:54:02.783864 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:02.783824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" event={"ID":"f3b576e7-c153-453e-95cf-547817cb24d2","Type":"ContainerStarted","Data":"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"}
Jul 29 01:54:02.784362 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:02.783890    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:54:02.803995 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:02.803940    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" podStartSLOduration=2.377721219 podStartE2EDuration="2.803924562s" totalImagesPullingTime="426.203343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:54:02.801958196 +0000 UTC m=+2497.233840676" watchObservedRunningTime="2026-07-29 01:54:02.803924562 +0000 UTC m=+2497.235807042"
Jul 29 01:54:08.802660 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:54:08.802630    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 01:57:26.309207 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:57:26.309180    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:57:26.323280 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:57:26.323258    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 01:57:26.536286 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:57:26.536253    2564 scope.go:117] "RemoveContainer" containerID="c5bf1431d6ee06085108452e31e3c54c9144e77bad00c951cc79bcd1085ea4a2"
Jul 29 01:58:26.558436 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:58:26.558398    2564 scope.go:117] "RemoveContainer" containerID="35ff70beef179a1b4da7108309602ce22078852c7d4940d98e77ea47493eb7aa"
Jul 29 01:59:26.581937 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:59:26.581860    2564 scope.go:117] "RemoveContainer" containerID="92f54ce51ff439749d2f46aae78c47f845be63bdd900ec9db7c3feca416056f1"
Jul 29 01:59:26.591174 ip-10-0-141-49 kubenswrapper[2564]: I0729 01:59:26.591139    2564 scope.go:117] "RemoveContainer" containerID="0e0d118c0561a80110061fbd6aeb501be1c2876b758e24f9ff0477144d253d0b"
Jul 29 02:00:00.141859 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.141826    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"]
Jul 29 02:00:00.145551 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.145517    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"
Jul 29 02:00:00.145715 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.145575    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"]
Jul 29 02:00:00.148284 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.148260    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:00:00.149462 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.149446    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"]
Jul 29 02:00:00.149577 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.149559    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"
Jul 29 02:00:00.153651 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.153629    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"
Jul 29 02:00:00.154498 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.154298    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"]
Jul 29 02:00:00.158478 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.158387    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"]
Jul 29 02:00:00.160253 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.160142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"
Jul 29 02:00:00.163576 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.163107    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"]
Jul 29 02:00:00.171792 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.171767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"
Jul 29 02:00:00.207575 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.207554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"
Jul 29 02:00:00.328711 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.328672    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"]
Jul 29 02:00:00.330639 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:00:00.330598    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c3cab41_a318_47fa_a1b3_28bece9dca55.slice/crio-e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d WatchSource:0}: Error finding container e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d: Status 404 returned error can't find the container with id e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d
Jul 29 02:00:00.370404 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.370299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"]
Jul 29 02:00:00.375603 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:00:00.375573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23b47bc_90e2_48fa_9d89_5c058baafcdb.slice/crio-3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171 WatchSource:0}: Error finding container 3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171: Status 404 returned error can't find the container with id 3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171
Jul 29 02:00:00.399290 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.399260    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"]
Jul 29 02:00:00.412248 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:00:00.412226    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3647fa6b_e15b_48fa_9eb5_58a53c6170b4.slice/crio-f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f WatchSource:0}: Error finding container f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f: Status 404 returned error can't find the container with id f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f
Jul 29 02:00:00.940939 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.940870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4" event={"ID":"3647fa6b-e15b-48fa-9eb5-58a53c6170b4","Type":"ContainerStarted","Data":"392d3346fb481537944f52f1f165196edc3c1c938f5fa201b61b809ead610628"}
Jul 29 02:00:00.940939 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.940923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4" event={"ID":"3647fa6b-e15b-48fa-9eb5-58a53c6170b4","Type":"ContainerStarted","Data":"f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f"}
Jul 29 02:00:00.942420 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.942397    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8" event={"ID":"7c3cab41-a318-47fa-a1b3-28bece9dca55","Type":"ContainerStarted","Data":"0a61f2651742081ae68b83cd6861ec84b875170790cd3ee1d87f4f2c5fb31222"}
Jul 29 02:00:00.942522 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.942446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8" event={"ID":"7c3cab41-a318-47fa-a1b3-28bece9dca55","Type":"ContainerStarted","Data":"e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d"}
Jul 29 02:00:00.943740 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.943720    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c" event={"ID":"a23b47bc-90e2-48fa-9d89-5c058baafcdb","Type":"ContainerStarted","Data":"9733f06084c5532544c5a8e81dc6ef13c968f2c7eb2233fa026274328225a03f"}
Jul 29 02:00:00.943818 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.943761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c" event={"ID":"a23b47bc-90e2-48fa-9d89-5c058baafcdb","Type":"ContainerStarted","Data":"3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171"}
Jul 29 02:00:00.956554 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.956515    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4" podStartSLOduration=0.956500108 podStartE2EDuration="956.500108ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:00:00.95549765 +0000 UTC m=+2855.387380132" watchObservedRunningTime="2026-07-29 02:00:00.956500108 +0000 UTC m=+2855.388382588"
Jul 29 02:00:00.972241 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.972199    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8" podStartSLOduration=0.972186552 podStartE2EDuration="972.186552ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:00:00.969352115 +0000 UTC m=+2855.401234595" watchObservedRunningTime="2026-07-29 02:00:00.972186552 +0000 UTC m=+2855.404069031"
Jul 29 02:00:00.985693 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:00.985654    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c" podStartSLOduration=0.985642254 podStartE2EDuration="985.642254ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:00:00.983114486 +0000 UTC m=+2855.414996969" watchObservedRunningTime="2026-07-29 02:00:00.985642254 +0000 UTC m=+2855.417524733"
Jul 29 02:00:01.951027 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.950996    2564 generic.go:356] "Generic (PLEG): container finished" podID="3647fa6b-e15b-48fa-9eb5-58a53c6170b4" containerID="392d3346fb481537944f52f1f165196edc3c1c938f5fa201b61b809ead610628" exitCode=0
Jul 29 02:00:01.951479 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.951072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4" event={"ID":"3647fa6b-e15b-48fa-9eb5-58a53c6170b4","Type":"ContainerDied","Data":"392d3346fb481537944f52f1f165196edc3c1c938f5fa201b61b809ead610628"}
Jul 29 02:00:01.952666 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.952613    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c3cab41-a318-47fa-a1b3-28bece9dca55" containerID="0a61f2651742081ae68b83cd6861ec84b875170790cd3ee1d87f4f2c5fb31222" exitCode=0
Jul 29 02:00:01.952776 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.952692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8" event={"ID":"7c3cab41-a318-47fa-a1b3-28bece9dca55","Type":"ContainerDied","Data":"0a61f2651742081ae68b83cd6861ec84b875170790cd3ee1d87f4f2c5fb31222"}
Jul 29 02:00:01.954196 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.954172    2564 generic.go:356] "Generic (PLEG): container finished" podID="a23b47bc-90e2-48fa-9d89-5c058baafcdb" containerID="9733f06084c5532544c5a8e81dc6ef13c968f2c7eb2233fa026274328225a03f" exitCode=0
Jul 29 02:00:01.954284 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:01.954256    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c" event={"ID":"a23b47bc-90e2-48fa-9d89-5c058baafcdb","Type":"ContainerDied","Data":"9733f06084c5532544c5a8e81dc6ef13c968f2c7eb2233fa026274328225a03f"}
Jul 29 02:00:03.120260 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.120233    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"
Jul 29 02:00:03.170301 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.170267    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"
Jul 29 02:00:03.173570 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.173543    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"
Jul 29 02:00:03.969164 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.969132    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"
Jul 29 02:00:03.969354 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.969174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c" event={"ID":"a23b47bc-90e2-48fa-9d89-5c058baafcdb","Type":"ContainerDied","Data":"3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171"}
Jul 29 02:00:03.969354 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.969223    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3efefe79332bbebb9941a4a84985379264a72a586782cafd2c5753bc685d5171"
Jul 29 02:00:03.971206 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.971177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4" event={"ID":"3647fa6b-e15b-48fa-9eb5-58a53c6170b4","Type":"ContainerDied","Data":"f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f"}
Jul 29 02:00:03.971357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.971223    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6fe827f24200c129ad238b3b4e42f67221f253404d3cc606755468cd350b81f"
Jul 29 02:00:03.971357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.971185    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754840-99lp4"
Jul 29 02:00:03.972611 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.972585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8" event={"ID":"7c3cab41-a318-47fa-a1b3-28bece9dca55","Type":"ContainerDied","Data":"e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d"}
Jul 29 02:00:03.972611 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.972611    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"
Jul 29 02:00:03.972803 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:00:03.972630    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1127c11ca1c059349f7787656b2cd3508ecd60e19424cf423446ed458c17d6d"
Jul 29 02:02:26.351701 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:26.351598    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 02:02:26.364375 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:26.364355    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 02:02:33.562364 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.562326    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051305264"]
Jul 29 02:02:33.562961 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.562940    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3647fa6b-e15b-48fa-9eb5-58a53c6170b4" containerName="cleanup"
Jul 29 02:02:33.562961 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.562963    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3647fa6b-e15b-48fa-9eb5-58a53c6170b4" containerName="cleanup"
Jul 29 02:02:33.563108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563003    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c3cab41-a318-47fa-a1b3-28bece9dca55" containerName="cleanup"
Jul 29 02:02:33.563108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563013    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c3cab41-a318-47fa-a1b3-28bece9dca55" containerName="cleanup"
Jul 29 02:02:33.563108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563032    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23b47bc-90e2-48fa-9d89-5c058baafcdb" containerName="cleanup"
Jul 29 02:02:33.563108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563041    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23b47bc-90e2-48fa-9d89-5c058baafcdb" containerName="cleanup"
Jul 29 02:02:33.563357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563169    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23b47bc-90e2-48fa-9d89-5c058baafcdb" containerName="cleanup"
Jul 29 02:02:33.563357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563186    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c3cab41-a318-47fa-a1b3-28bece9dca55" containerName="cleanup"
Jul 29 02:02:33.563357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.563201    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3647fa6b-e15b-48fa-9eb5-58a53c6170b4" containerName="cleanup"
Jul 29 02:02:33.566453 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.566433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:33.569224 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.569177    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tfngb\""
Jul 29 02:02:33.571983 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.571948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051305264"]
Jul 29 02:02:33.585943 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.585917    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trnlm\" (UniqueName: \"kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm\") pod \"test-curl-7560-140613051305264\" (UID: \"459df616-ffaa-45c8-bb7c-8997520de737\") " pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:33.687286 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.687264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trnlm\" (UniqueName: \"kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm\") pod \"test-curl-7560-140613051305264\" (UID: \"459df616-ffaa-45c8-bb7c-8997520de737\") " pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:33.695799 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.695773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trnlm\" (UniqueName: \"kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm\") pod \"test-curl-7560-140613051305264\" (UID: \"459df616-ffaa-45c8-bb7c-8997520de737\") " pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:33.879304 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:33.879228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:34.017322 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:34.017297    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051305264"]
Jul 29 02:02:34.019221 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:34.019193    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod459df616_ffaa_45c8_bb7c_8997520de737.slice/crio-aeb245bfbd99a74cbb00de76417b7897583c42a5a5b8e8ba85a96fd880394a7e WatchSource:0}: Error finding container aeb245bfbd99a74cbb00de76417b7897583c42a5a5b8e8ba85a96fd880394a7e: Status 404 returned error can't find the container with id aeb245bfbd99a74cbb00de76417b7897583c42a5a5b8e8ba85a96fd880394a7e
Jul 29 02:02:34.020437 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:34.020417    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 02:02:34.890834 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:34.890808    2564 generic.go:356] "Generic (PLEG): container finished" podID="459df616-ffaa-45c8-bb7c-8997520de737" containerID="3e20338e32a29a3bb17d9179253884ddd2d7a0e55a994e4590868a6f0ef43141" exitCode=0
Jul 29 02:02:34.891135 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:34.890892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051305264" event={"ID":"459df616-ffaa-45c8-bb7c-8997520de737","Type":"ContainerDied","Data":"3e20338e32a29a3bb17d9179253884ddd2d7a0e55a994e4590868a6f0ef43141"}
Jul 29 02:02:34.891135 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:34.890932    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051305264" event={"ID":"459df616-ffaa-45c8-bb7c-8997520de737","Type":"ContainerStarted","Data":"aeb245bfbd99a74cbb00de76417b7897583c42a5a5b8e8ba85a96fd880394a7e"}
Jul 29 02:02:36.044430 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.044407    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:36.079388 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.079360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051305264_459df616-ffaa-45c8-bb7c-8997520de737/test-curl-7560-140613051305264/0.log"
Jul 29 02:02:36.103760 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.103736    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051305264"]
Jul 29 02:02:36.106752 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.106729    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051305264"]
Jul 29 02:02:36.109346 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.109330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trnlm\" (UniqueName: \"kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm\") pod \"459df616-ffaa-45c8-bb7c-8997520de737\" (UID: \"459df616-ffaa-45c8-bb7c-8997520de737\") "
Jul 29 02:02:36.111364 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.111333    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm" (OuterVolumeSpecName: "kube-api-access-trnlm") pod "459df616-ffaa-45c8-bb7c-8997520de737" (UID: "459df616-ffaa-45c8-bb7c-8997520de737"). InnerVolumeSpecName "kube-api-access-trnlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:36.118160 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.118082    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="459df616-ffaa-45c8-bb7c-8997520de737" path="/var/lib/kubelet/pods/459df616-ffaa-45c8-bb7c-8997520de737/volumes"
Jul 29 02:02:36.210489 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.210434    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trnlm\" (UniqueName: \"kubernetes.io/projected/459df616-ffaa-45c8-bb7c-8997520de737-kube-api-access-trnlm\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:36.225217 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.225192    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051360432"]
Jul 29 02:02:36.225592 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.225580    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="459df616-ffaa-45c8-bb7c-8997520de737" containerName="test-curl-7560-140613051305264"
Jul 29 02:02:36.225636 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.225593    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="459df616-ffaa-45c8-bb7c-8997520de737" containerName="test-curl-7560-140613051305264"
Jul 29 02:02:36.225694 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.225684    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="459df616-ffaa-45c8-bb7c-8997520de737" containerName="test-curl-7560-140613051305264"
Jul 29 02:02:36.227994 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.227977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:36.234470 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.234448    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051360432"]
Jul 29 02:02:36.311449 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.311424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg2cs\" (UniqueName: \"kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs\") pod \"test-curl-7560-140613051360432\" (UID: \"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf\") " pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:36.412709 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.412683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg2cs\" (UniqueName: \"kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs\") pod \"test-curl-7560-140613051360432\" (UID: \"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf\") " pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:36.421224 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.421200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg2cs\" (UniqueName: \"kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs\") pod \"test-curl-7560-140613051360432\" (UID: \"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf\") " pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:36.538476 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.538450    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:36.674385 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.674358    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051360432"]
Jul 29 02:02:36.675862 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:36.675828    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6e262cd_a1ea_4a8a_a51b_7197bceab5cf.slice/crio-46bd6024bdfb68b0ddb286685f07f56b76b96361b400cc01cfb72a93446b6deb WatchSource:0}: Error finding container 46bd6024bdfb68b0ddb286685f07f56b76b96361b400cc01cfb72a93446b6deb: Status 404 returned error can't find the container with id 46bd6024bdfb68b0ddb286685f07f56b76b96361b400cc01cfb72a93446b6deb
Jul 29 02:02:36.904057 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.903979    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051305264"
Jul 29 02:02:36.904057 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.904016    2564 scope.go:117] "RemoveContainer" containerID="3e20338e32a29a3bb17d9179253884ddd2d7a0e55a994e4590868a6f0ef43141"
Jul 29 02:02:36.905530 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:36.905503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051360432" event={"ID":"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf","Type":"ContainerStarted","Data":"46bd6024bdfb68b0ddb286685f07f56b76b96361b400cc01cfb72a93446b6deb"}
Jul 29 02:02:37.912820 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:37.912782    2564 generic.go:356] "Generic (PLEG): container finished" podID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" containerID="8fe5362857186ca9c8e9a5baee3e976b1c48ed8c256b5fcc021fb5887a0fdfaf" exitCode=0
Jul 29 02:02:37.913256 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:37.912874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051360432" event={"ID":"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf","Type":"ContainerDied","Data":"8fe5362857186ca9c8e9a5baee3e976b1c48ed8c256b5fcc021fb5887a0fdfaf"}
Jul 29 02:02:39.056705 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.056681    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:39.075667 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.075641    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051360432_f6e262cd-a1ea-4a8a-a51b-7197bceab5cf/test-curl-7560-140613051360432/0.log"
Jul 29 02:02:39.101350 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.101325    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051360432"]
Jul 29 02:02:39.104022 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.104000    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051360432"]
Jul 29 02:02:39.138007 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.137985    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg2cs\" (UniqueName: \"kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs\") pod \"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf\" (UID: \"f6e262cd-a1ea-4a8a-a51b-7197bceab5cf\") "
Jul 29 02:02:39.139971 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.139947    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs" (OuterVolumeSpecName: "kube-api-access-zg2cs") pod "f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" (UID: "f6e262cd-a1ea-4a8a-a51b-7197bceab5cf"). InnerVolumeSpecName "kube-api-access-zg2cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:39.238882 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.238825    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg2cs\" (UniqueName: \"kubernetes.io/projected/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf-kube-api-access-zg2cs\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:39.254659 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.254634    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:39.255035 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.255023    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" containerName="test-curl-7560-140613051360432"
Jul 29 02:02:39.255078 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.255037    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" containerName="test-curl-7560-140613051360432"
Jul 29 02:02:39.255122 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.255112    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" containerName="test-curl-7560-140613051360432"
Jul 29 02:02:39.257914 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.257900    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:39.264331 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.264310    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:39.339626 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.339601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5m2\" (UniqueName: \"kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2\") pod \"test-curl-7560-140613051204928\" (UID: \"3ea3447c-614d-4e96-8996-09ad5e3cfeb9\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:39.440261 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.440231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5m2\" (UniqueName: \"kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2\") pod \"test-curl-7560-140613051204928\" (UID: \"3ea3447c-614d-4e96-8996-09ad5e3cfeb9\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:39.448066 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.448037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5m2\" (UniqueName: \"kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2\") pod \"test-curl-7560-140613051204928\" (UID: \"3ea3447c-614d-4e96-8996-09ad5e3cfeb9\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:39.568952 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.568927    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:39.703021 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.702996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:39.705081 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:39.705055    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ea3447c_614d_4e96_8996_09ad5e3cfeb9.slice/crio-2f06557ee75d4ca7fd10969346ab2066cf740fa5ff6388744e58542cb5027985 WatchSource:0}: Error finding container 2f06557ee75d4ca7fd10969346ab2066cf740fa5ff6388744e58542cb5027985: Status 404 returned error can't find the container with id 2f06557ee75d4ca7fd10969346ab2066cf740fa5ff6388744e58542cb5027985
Jul 29 02:02:39.933741 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.933661    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46bd6024bdfb68b0ddb286685f07f56b76b96361b400cc01cfb72a93446b6deb"
Jul 29 02:02:39.933741 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.933664    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:39.934903 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:39.934876    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051204928" event={"ID":"3ea3447c-614d-4e96-8996-09ad5e3cfeb9","Type":"ContainerStarted","Data":"2f06557ee75d4ca7fd10969346ab2066cf740fa5ff6388744e58542cb5027985"}
Jul 29 02:02:39.944623 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:02:39.944597    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7560-140613051360432\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" pod="opendatahub/test-curl-7560-140613051360432"
Jul 29 02:02:40.117893 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:40.117866    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e262cd-a1ea-4a8a-a51b-7197bceab5cf" path="/var/lib/kubelet/pods/f6e262cd-a1ea-4a8a-a51b-7197bceab5cf/volumes"
Jul 29 02:02:40.942619 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:40.942586    2564 generic.go:356] "Generic (PLEG): container finished" podID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" containerID="58d75cbf49d222a0c3b32ad243624413bc1b00b564ae094c5ed20900473e9f66" exitCode=0
Jul 29 02:02:40.942772 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:40.942670    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051204928" event={"ID":"3ea3447c-614d-4e96-8996-09ad5e3cfeb9","Type":"ContainerDied","Data":"58d75cbf49d222a0c3b32ad243624413bc1b00b564ae094c5ed20900473e9f66"}
Jul 29 02:02:42.084509 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.084485    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.102977 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.102951    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051204928_3ea3447c-614d-4e96-8996-09ad5e3cfeb9/test-curl-7560-140613051204928/0.log"
Jul 29 02:02:42.123230 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.123202    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:42.126834 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.126811    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:42.160411 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.160382    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql5m2\" (UniqueName: \"kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2\") pod \"3ea3447c-614d-4e96-8996-09ad5e3cfeb9\" (UID: \"3ea3447c-614d-4e96-8996-09ad5e3cfeb9\") "
Jul 29 02:02:42.162383 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.162359    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2" (OuterVolumeSpecName: "kube-api-access-ql5m2") pod "3ea3447c-614d-4e96-8996-09ad5e3cfeb9" (UID: "3ea3447c-614d-4e96-8996-09ad5e3cfeb9"). InnerVolumeSpecName "kube-api-access-ql5m2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:42.244720 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.244658    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:42.245074 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.245060    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:42.245123 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.245076    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:42.245192 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.245179    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:42.248102 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.248087    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.254953 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.254662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:42.261889 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.261869    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql5m2\" (UniqueName: \"kubernetes.io/projected/3ea3447c-614d-4e96-8996-09ad5e3cfeb9-kube-api-access-ql5m2\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:42.362393 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.362356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9f4j\" (UniqueName: \"kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j\") pod \"test-curl-7560-140613051204928\" (UID: \"2348c41b-3037-4711-aa61-4b8cf56fee61\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.464019 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.463979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9f4j\" (UniqueName: \"kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j\") pod \"test-curl-7560-140613051204928\" (UID: \"2348c41b-3037-4711-aa61-4b8cf56fee61\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.472283 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.472252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9f4j\" (UniqueName: \"kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j\") pod \"test-curl-7560-140613051204928\" (UID: \"2348c41b-3037-4711-aa61-4b8cf56fee61\") " pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.559393 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.559364    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.695205 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.695182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:42.696697 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:42.696669    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2348c41b_3037_4711_aa61_4b8cf56fee61.slice/crio-20c93e2a411fc9889215cd6d72e5f0ea912d5d4524ec6ebf7bcc5813d5e8e451 WatchSource:0}: Error finding container 20c93e2a411fc9889215cd6d72e5f0ea912d5d4524ec6ebf7bcc5813d5e8e451: Status 404 returned error can't find the container with id 20c93e2a411fc9889215cd6d72e5f0ea912d5d4524ec6ebf7bcc5813d5e8e451
Jul 29 02:02:42.956032 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.955963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:42.956032 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.955986    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f06557ee75d4ca7fd10969346ab2066cf740fa5ff6388744e58542cb5027985"
Jul 29 02:02:42.957289 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.957265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051204928" event={"ID":"2348c41b-3037-4711-aa61-4b8cf56fee61","Type":"ContainerStarted","Data":"20c93e2a411fc9889215cd6d72e5f0ea912d5d4524ec6ebf7bcc5813d5e8e451"}
Jul 29 02:02:42.968316 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:42.968289    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7560-140613051204928" oldPodUID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61"
Jul 29 02:02:43.965574 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:43.965537    2564 generic.go:356] "Generic (PLEG): container finished" podID="2348c41b-3037-4711-aa61-4b8cf56fee61" containerID="99bed43e8e58a7f52cf76923c17b5fc3700c08a90c8e2aff5ef8ce42e09cabcf" exitCode=0
Jul 29 02:02:43.965970 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:43.965624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051204928" event={"ID":"2348c41b-3037-4711-aa61-4b8cf56fee61","Type":"ContainerDied","Data":"99bed43e8e58a7f52cf76923c17b5fc3700c08a90c8e2aff5ef8ce42e09cabcf"}
Jul 29 02:02:44.117356 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:44.117325    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ea3447c-614d-4e96-8996-09ad5e3cfeb9" path="/var/lib/kubelet/pods/3ea3447c-614d-4e96-8996-09ad5e3cfeb9/volumes"
Jul 29 02:02:45.109831 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.109808    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:45.128998 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.128974    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051204928_2348c41b-3037-4711-aa61-4b8cf56fee61/test-curl-7560-140613051204928/0.log"
Jul 29 02:02:45.153488 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.153462    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:45.157946 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.157924    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051204928"]
Jul 29 02:02:45.188281 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.188259    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9f4j\" (UniqueName: \"kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j\") pod \"2348c41b-3037-4711-aa61-4b8cf56fee61\" (UID: \"2348c41b-3037-4711-aa61-4b8cf56fee61\") "
Jul 29 02:02:45.190263 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.190243    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j" (OuterVolumeSpecName: "kube-api-access-s9f4j") pod "2348c41b-3037-4711-aa61-4b8cf56fee61" (UID: "2348c41b-3037-4711-aa61-4b8cf56fee61"). InnerVolumeSpecName "kube-api-access-s9f4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:45.280494 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.280471    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:45.280910 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.280895    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:45.280950 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.280911    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:45.281022 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.281012    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" containerName="test-curl-7560-140613051204928"
Jul 29 02:02:45.284091 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.284068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:45.288331 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.288308    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:45.289598 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.289568    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9f4j\" (UniqueName: \"kubernetes.io/projected/2348c41b-3037-4711-aa61-4b8cf56fee61-kube-api-access-s9f4j\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:45.391774 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.391186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8b8s\" (UniqueName: \"kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s\") pod \"test-curl-7560-140613051206800\" (UID: \"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:45.492636 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.492610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8b8s\" (UniqueName: \"kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s\") pod \"test-curl-7560-140613051206800\" (UID: \"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:45.500015 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.499989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8b8s\" (UniqueName: \"kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s\") pod \"test-curl-7560-140613051206800\" (UID: \"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:45.595734 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.595664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:45.733357 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.733333    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:45.734843 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:45.734804    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b5f9e63_fc27_4010_87c7_0ce1e3bd19a1.slice/crio-b775a297fb14a77a48c1f93478c8784883306858aeb712136a059dea899daaed WatchSource:0}: Error finding container b775a297fb14a77a48c1f93478c8784883306858aeb712136a059dea899daaed: Status 404 returned error can't find the container with id b775a297fb14a77a48c1f93478c8784883306858aeb712136a059dea899daaed
Jul 29 02:02:45.983354 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.983272    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:45.983354 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.983317    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20c93e2a411fc9889215cd6d72e5f0ea912d5d4524ec6ebf7bcc5813d5e8e451"
Jul 29 02:02:45.984571 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:45.984542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051206800" event={"ID":"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1","Type":"ContainerStarted","Data":"b775a297fb14a77a48c1f93478c8784883306858aeb712136a059dea899daaed"}
Jul 29 02:02:45.994526 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:02:45.994499    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7560-140613051204928\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:46.115942 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:02:46.115915    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7560-140613051204928\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" pod="opendatahub/test-curl-7560-140613051204928"
Jul 29 02:02:46.117656 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:46.117635    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2348c41b-3037-4711-aa61-4b8cf56fee61" path="/var/lib/kubelet/pods/2348c41b-3037-4711-aa61-4b8cf56fee61/volumes"
Jul 29 02:02:46.991762 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:46.991731    2564 generic.go:356] "Generic (PLEG): container finished" podID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" containerID="cab08e1af38cc25b222f3b6695e234e9b513cc3286b33150db82b58c1b1a7d47" exitCode=0
Jul 29 02:02:46.991965 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:46.991770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051206800" event={"ID":"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1","Type":"ContainerDied","Data":"cab08e1af38cc25b222f3b6695e234e9b513cc3286b33150db82b58c1b1a7d47"}
Jul 29 02:02:48.134457 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.134431    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.154497 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.154467    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051206800_0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1/test-curl-7560-140613051206800/0.log"
Jul 29 02:02:48.178164 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.178120    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:48.182631 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.182609    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:48.218470 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.218448    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8b8s\" (UniqueName: \"kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s\") pod \"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1\" (UID: \"0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1\") "
Jul 29 02:02:48.220470 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.220447    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s" (OuterVolumeSpecName: "kube-api-access-j8b8s") pod "0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" (UID: "0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1"). InnerVolumeSpecName "kube-api-access-j8b8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:48.296040 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.296010    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:48.296449 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.296435    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:48.296501 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.296451    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:48.296558 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.296518    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:48.299681 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.299655    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.303120 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.303093    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:48.320108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.320087    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8b8s\" (UniqueName: \"kubernetes.io/projected/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1-kube-api-access-j8b8s\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:48.420648 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.420624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9mxv\" (UniqueName: \"kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv\") pod \"test-curl-7560-140613051206800\" (UID: \"1e8ab462-2fd4-46c7-9803-5defbce90f4b\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.521664 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.521627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9mxv\" (UniqueName: \"kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv\") pod \"test-curl-7560-140613051206800\" (UID: \"1e8ab462-2fd4-46c7-9803-5defbce90f4b\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.531994 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.531969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9mxv\" (UniqueName: \"kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv\") pod \"test-curl-7560-140613051206800\" (UID: \"1e8ab462-2fd4-46c7-9803-5defbce90f4b\") " pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.609689 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.609631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:48.744119 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:48.744095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:48.746046 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:48.746021    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e8ab462_2fd4_46c7_9803_5defbce90f4b.slice/crio-5379f114acf8fe448691c1d3ad01fe0948fe61e59f99c755300f0e60397a05a5 WatchSource:0}: Error finding container 5379f114acf8fe448691c1d3ad01fe0948fe61e59f99c755300f0e60397a05a5: Status 404 returned error can't find the container with id 5379f114acf8fe448691c1d3ad01fe0948fe61e59f99c755300f0e60397a05a5
Jul 29 02:02:49.006411 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:49.006329    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:49.006587 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:49.006340    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b775a297fb14a77a48c1f93478c8784883306858aeb712136a059dea899daaed"
Jul 29 02:02:49.007629 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:49.007605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051206800" event={"ID":"1e8ab462-2fd4-46c7-9803-5defbce90f4b","Type":"ContainerStarted","Data":"5379f114acf8fe448691c1d3ad01fe0948fe61e59f99c755300f0e60397a05a5"}
Jul 29 02:02:49.018120 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:49.018097    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7560-140613051206800" oldPodUID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b"
Jul 29 02:02:50.015521 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:50.015490    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" containerID="e7db8037a7bc7370c94945b4451d9f8e49674283792a8847cca8ec5fd7124037" exitCode=0
Jul 29 02:02:50.015853 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:50.015543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7560-140613051206800" event={"ID":"1e8ab462-2fd4-46c7-9803-5defbce90f4b","Type":"ContainerDied","Data":"e7db8037a7bc7370c94945b4451d9f8e49674283792a8847cca8ec5fd7124037"}
Jul 29 02:02:50.120986 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:50.120954    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1" path="/var/lib/kubelet/pods/0b5f9e63-fc27-4010-87c7-0ce1e3bd19a1/volumes"
Jul 29 02:02:51.160397 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.160373    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:51.180440 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.180410    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7560-140613051206800_1e8ab462-2fd4-46c7-9803-5defbce90f4b/test-curl-7560-140613051206800/0.log"
Jul 29 02:02:51.207306 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.207284    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:51.211420 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.211395    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7560-140613051206800"]
Jul 29 02:02:51.246756 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.246735    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9mxv\" (UniqueName: \"kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv\") pod \"1e8ab462-2fd4-46c7-9803-5defbce90f4b\" (UID: \"1e8ab462-2fd4-46c7-9803-5defbce90f4b\") "
Jul 29 02:02:51.248799 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.248778    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv" (OuterVolumeSpecName: "kube-api-access-z9mxv") pod "1e8ab462-2fd4-46c7-9803-5defbce90f4b" (UID: "1e8ab462-2fd4-46c7-9803-5defbce90f4b"). InnerVolumeSpecName "kube-api-access-z9mxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:02:51.348454 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:51.348430    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9mxv\" (UniqueName: \"kubernetes.io/projected/1e8ab462-2fd4-46c7-9803-5defbce90f4b-kube-api-access-z9mxv\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:02:52.031412 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:52.031381    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:52.031587 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:52.031417    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5379f114acf8fe448691c1d3ad01fe0948fe61e59f99c755300f0e60397a05a5"
Jul 29 02:02:52.041548 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:02:52.041516    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7560-140613051206800\" is forbidden: User \"system:node:ip-10-0-141-49.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-49.ec2.internal' and this object" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" pod="opendatahub/test-curl-7560-140613051206800"
Jul 29 02:02:52.118100 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:52.118056    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" path="/var/lib/kubelet/pods/1e8ab462-2fd4-46c7-9803-5defbce90f4b/volumes"
Jul 29 02:02:54.876349 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.876314    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:02:54.876968 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.876949    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:54.876968 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.876970    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:54.877129 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.877086    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8ab462-2fd4-46c7-9803-5defbce90f4b" containerName="test-curl-7560-140613051206800"
Jul 29 02:02:54.881054 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.881033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:54.883543 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.883519    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-59f277-serving-cert\""
Jul 29 02:02:54.890016 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.889557    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:02:54.984265 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.984223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:54.984422 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.984328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dchn\" (UniqueName: \"kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:54.984480 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:54.984383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.086008 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.085975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.086008 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.086015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dchn\" (UniqueName: \"kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.086311 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.086034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.086741 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.086712    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.088483 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.088462    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.094905 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.094879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dchn\" (UniqueName: \"kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn\") pod \"maas-api-e2e-ipp-a-59f277-767657f877-z2d5l\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.193291 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.193234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:55.341911 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:55.341886    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:02:55.342970 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:02:55.342945    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072c7f4c_c3cd_414a_8410_93fdba753d8f.slice/crio-5947d290e00d3f14438458810b25d7955abc1005a47707702afe8ce8e6fc0975 WatchSource:0}: Error finding container 5947d290e00d3f14438458810b25d7955abc1005a47707702afe8ce8e6fc0975: Status 404 returned error can't find the container with id 5947d290e00d3f14438458810b25d7955abc1005a47707702afe8ce8e6fc0975
Jul 29 02:02:56.056016 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:56.055934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" event={"ID":"072c7f4c-c3cd-414a-8410-93fdba753d8f","Type":"ContainerStarted","Data":"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"}
Jul 29 02:02:56.056016 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:56.055995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" event={"ID":"072c7f4c-c3cd-414a-8410-93fdba753d8f","Type":"ContainerStarted","Data":"5947d290e00d3f14438458810b25d7955abc1005a47707702afe8ce8e6fc0975"}
Jul 29 02:02:56.056460 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:56.056028    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:02:56.075848 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:02:56.075797    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" podStartSLOduration=1.628757096 podStartE2EDuration="2.075780565s" totalImagesPullingTime="447.023469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:02:56.073131693 +0000 UTC m=+3030.505014178" watchObservedRunningTime="2026-07-29 02:02:56.075780565 +0000 UTC m=+3030.507663044"
Jul 29 02:03:02.069605 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:02.069577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:03:24.051852 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.051762    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:03:24.056703 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.056676    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.059202 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.059182    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-7f1f73-serving-cert\""
Jul 29 02:03:24.065971 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.065949    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:03:24.131967 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.131939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8rkh\" (UniqueName: \"kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.132126 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.131972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.132126 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.132115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.233419 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.233376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.233616 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.233453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8rkh\" (UniqueName: \"kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.233616 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.233488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.234108 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.234089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.235772 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.235753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.244050 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.244029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8rkh\" (UniqueName: \"kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh\") pod \"maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.372101 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.372027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:24.512601 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:24.512569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:03:24.514233 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:03:24.514203    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34f4f25_2978_462b_8137_54fc32321db3.slice/crio-2ba9e880f8b07e9f080384dd87a93d9e04499e44b249d0629025f6887cbf15e2 WatchSource:0}: Error finding container 2ba9e880f8b07e9f080384dd87a93d9e04499e44b249d0629025f6887cbf15e2: Status 404 returned error can't find the container with id 2ba9e880f8b07e9f080384dd87a93d9e04499e44b249d0629025f6887cbf15e2
Jul 29 02:03:25.242304 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:25.242266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" event={"ID":"b34f4f25-2978-462b-8137-54fc32321db3","Type":"ContainerStarted","Data":"2ba9e880f8b07e9f080384dd87a93d9e04499e44b249d0629025f6887cbf15e2"}
Jul 29 02:03:26.250052 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:26.250018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" event={"ID":"b34f4f25-2978-462b-8137-54fc32321db3","Type":"ContainerStarted","Data":"0cabc61af5dd6448601f09d1705a3cc0dcda3157464d18b13a0dc679f69dee2a"}
Jul 29 02:03:26.250453 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:26.250303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:26.267941 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:26.267890    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" podStartSLOduration=0.756135085 podStartE2EDuration="2.267874827s" totalImagesPullingTime="1.511739742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:03:26.266691866 +0000 UTC m=+3060.698574359" watchObservedRunningTime="2026-07-29 02:03:26.267874827 +0000 UTC m=+3060.699757306"
Jul 29 02:03:32.261647 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:32.261618    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:03:48.032768 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.032732    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:03:48.036600 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.036584    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:48.064280 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.064255    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:03:48.154555 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.154525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgfw\" (UniqueName: \"kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw\") pod \"kuadrant-operator-catalog-dpf4b\" (UID: \"26e58bc3-ef1f-45a5-916a-6e3cf330d637\") " pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:48.255250 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.255215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgfw\" (UniqueName: \"kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw\") pod \"kuadrant-operator-catalog-dpf4b\" (UID: \"26e58bc3-ef1f-45a5-916a-6e3cf330d637\") " pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:48.264599 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.264574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgfw\" (UniqueName: \"kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw\") pod \"kuadrant-operator-catalog-dpf4b\" (UID: \"26e58bc3-ef1f-45a5-916a-6e3cf330d637\") " pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:48.345669 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.345605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:48.692675 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:48.692647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:03:48.694281 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:03:48.694254    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e58bc3_ef1f_45a5_916a_6e3cf330d637.slice/crio-76c41939719434ad41d20b10e952c6b98664adb710103cf72d00a3222411541f WatchSource:0}: Error finding container 76c41939719434ad41d20b10e952c6b98664adb710103cf72d00a3222411541f: Status 404 returned error can't find the container with id 76c41939719434ad41d20b10e952c6b98664adb710103cf72d00a3222411541f
Jul 29 02:03:49.405772 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:49.405689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" event={"ID":"26e58bc3-ef1f-45a5-916a-6e3cf330d637","Type":"ContainerStarted","Data":"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"}
Jul 29 02:03:49.405772 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:49.405745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" event={"ID":"26e58bc3-ef1f-45a5-916a-6e3cf330d637","Type":"ContainerStarted","Data":"76c41939719434ad41d20b10e952c6b98664adb710103cf72d00a3222411541f"}
Jul 29 02:03:49.425556 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:49.425506    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" podStartSLOduration=1.081377812 podStartE2EDuration="1.425488137s" totalImagesPullingTime="344.110325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:03:49.42166148 +0000 UTC m=+3083.853543961" watchObservedRunningTime="2026-07-29 02:03:49.425488137 +0000 UTC m=+3083.857370663"
Jul 29 02:03:58.346050 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:58.346006    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:58.346586 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:58.346082    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:58.368037 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:58.368009    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:58.482305 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:58.482281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:03:58.594842 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:03:58.594808    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:04:00.474179 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:00.474124    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" podUID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" containerName="registry-server" containerID="cri-o://3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb" gracePeriod=2
Jul 29 02:04:00.738265 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:00.738211    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:04:00.868672 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:00.868647    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpgfw\" (UniqueName: \"kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw\") pod \"26e58bc3-ef1f-45a5-916a-6e3cf330d637\" (UID: \"26e58bc3-ef1f-45a5-916a-6e3cf330d637\") "
Jul 29 02:04:00.870697 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:00.870650    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw" (OuterVolumeSpecName: "kube-api-access-rpgfw") pod "26e58bc3-ef1f-45a5-916a-6e3cf330d637" (UID: "26e58bc3-ef1f-45a5-916a-6e3cf330d637"). InnerVolumeSpecName "kube-api-access-rpgfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:04:00.969511 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:00.969490    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpgfw\" (UniqueName: \"kubernetes.io/projected/26e58bc3-ef1f-45a5-916a-6e3cf330d637-kube-api-access-rpgfw\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:04:01.481779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.481740    2564 generic.go:356] "Generic (PLEG): container finished" podID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" containerID="3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb" exitCode=0
Jul 29 02:04:01.482277 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.481784    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b"
Jul 29 02:04:01.482277 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.481825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" event={"ID":"26e58bc3-ef1f-45a5-916a-6e3cf330d637","Type":"ContainerDied","Data":"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"}
Jul 29 02:04:01.482277 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.481888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dpf4b" event={"ID":"26e58bc3-ef1f-45a5-916a-6e3cf330d637","Type":"ContainerDied","Data":"76c41939719434ad41d20b10e952c6b98664adb710103cf72d00a3222411541f"}
Jul 29 02:04:01.482277 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.481918    2564 scope.go:117] "RemoveContainer" containerID="3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"
Jul 29 02:04:01.492789 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.492772    2564 scope.go:117] "RemoveContainer" containerID="3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"
Jul 29 02:04:01.493045 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:04:01.493025    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb\": container with ID starting with 3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb not found: ID does not exist" containerID="3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"
Jul 29 02:04:01.493117 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.493053    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb"} err="failed to get container status \"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb\": rpc error: code = NotFound desc = could not find container \"3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb\": container with ID starting with 3cfbd949de374109271f03fa4d2da524c1fe8f62ec41aaa8c8448a64413967eb not found: ID does not exist"
Jul 29 02:04:01.504492 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.504462    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:04:01.507587 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:01.507567    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dpf4b"]
Jul 29 02:04:02.117941 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:02.117909    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" path="/var/lib/kubelet/pods/26e58bc3-ef1f-45a5-916a-6e3cf330d637/volumes"
Jul 29 02:04:57.571102 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.571014    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:04:57.571656 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.571574    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" containerName="registry-server"
Jul 29 02:04:57.571656 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.571591    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" containerName="registry-server"
Jul 29 02:04:57.571771 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.571674    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e58bc3-ef1f-45a5-916a-6e3cf330d637" containerName="registry-server"
Jul 29 02:04:57.575278 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.575251    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.583374 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.583354    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-382dad-serving-cert\""
Jul 29 02:04:57.588734 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.588710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:04:57.674386 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.674342    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.674572 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.674464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdz66\" (UniqueName: \"kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.674572 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.674550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.775879 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.775842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.776042 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.776003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdz66\" (UniqueName: \"kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.776097 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.776069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.776502 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.776485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.778651 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.778629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.787298 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.787274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdz66\" (UniqueName: \"kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66\") pod \"maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:57.886472 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:57.886419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:58.243411 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.243381    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:04:58.244249 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:04:58.244218    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d5a25e_0d33_4830_a806_3cfee26378e0.slice/crio-51a46bce20ebf877eff2558da53f8f7a7be30ac2de5ef6de24ecefae81f80913 WatchSource:0}: Error finding container 51a46bce20ebf877eff2558da53f8f7a7be30ac2de5ef6de24ecefae81f80913: Status 404 returned error can't find the container with id 51a46bce20ebf877eff2558da53f8f7a7be30ac2de5ef6de24ecefae81f80913
Jul 29 02:04:58.370004 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.369979    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:04:58.375055 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.375037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.379219 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.379197    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-382dad\""
Jul 29 02:04:58.379335 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.379241    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-382dad-dockercfg-x8j2x\""
Jul 29 02:04:58.386418 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.386392    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:04:58.485470 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.485434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6brp9\" (UniqueName: \"kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.485627 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.485537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.586254 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.586224    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6brp9\" (UniqueName: \"kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.586632 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.586281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.587199 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.587170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.597846 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.597821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6brp9\" (UniqueName: \"kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9\") pod \"payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.687865 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.687843    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:04:58.832405 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.832379    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:04:58.833828 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:04:58.833801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb534343_679b_411b_8106_a04640b1114b.slice/crio-e81c6d0b372a743b8c7b8046cda4d1c5700730ac391a14b1e2b52ac79c4f508a WatchSource:0}: Error finding container e81c6d0b372a743b8c7b8046cda4d1c5700730ac391a14b1e2b52ac79c4f508a: Status 404 returned error can't find the container with id e81c6d0b372a743b8c7b8046cda4d1c5700730ac391a14b1e2b52ac79c4f508a
Jul 29 02:04:58.847816 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.847790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" event={"ID":"db534343-679b-411b-8106-a04640b1114b","Type":"ContainerStarted","Data":"e81c6d0b372a743b8c7b8046cda4d1c5700730ac391a14b1e2b52ac79c4f508a"}
Jul 29 02:04:58.849230 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.849204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" event={"ID":"d2d5a25e-0d33-4830-a806-3cfee26378e0","Type":"ContainerStarted","Data":"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"}
Jul 29 02:04:58.849302 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.849254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" event={"ID":"d2d5a25e-0d33-4830-a806-3cfee26378e0","Type":"ContainerStarted","Data":"51a46bce20ebf877eff2558da53f8f7a7be30ac2de5ef6de24ecefae81f80913"}
Jul 29 02:04:58.849356 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.849337    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:04:58.871644 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:04:58.871595    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" podStartSLOduration=1.466725189 podStartE2EDuration="1.871576359s" totalImagesPullingTime="404.85117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:04:58.866281 +0000 UTC m=+3153.298163481" watchObservedRunningTime="2026-07-29 02:04:58.871576359 +0000 UTC m=+3153.303458840"
Jul 29 02:05:00.866489 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:00.866447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" event={"ID":"db534343-679b-411b-8106-a04640b1114b","Type":"ContainerStarted","Data":"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"}
Jul 29 02:05:00.866860 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:00.866545    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:05:00.887670 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:00.887630    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" podStartSLOduration=0.941385974 podStartE2EDuration="2.887615922s" totalImagesPullingTime="1.946229948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:05:00.884051602 +0000 UTC m=+3155.315934082" watchObservedRunningTime="2026-07-29 02:05:00.887615922 +0000 UTC m=+3155.319498401"
Jul 29 02:05:04.863563 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:04.863530    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:05:11.876245 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:11.876211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:05:23.121127 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.121086    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"]
Jul 29 02:05:23.124081 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.124058    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:23.127047 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.127029    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:05:23.132806 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.132773    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"]
Jul 29 02:05:23.211688 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.211662    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd\" (UID: \"f7531c82-0d7d-4ebc-a3c6-5821fd692f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:23.312817 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.312790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd\" (UID: \"f7531c82-0d7d-4ebc-a3c6-5821fd692f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:23.313451 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.313431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd\" (UID: \"f7531c82-0d7d-4ebc-a3c6-5821fd692f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:23.435535 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.435477    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:23.580959 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:05:23.580928    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7531c82_0d7d_4ebc_a3c6_5821fd692f04.slice/crio-6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb WatchSource:0}: Error finding container 6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb: Status 404 returned error can't find the container with id 6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb
Jul 29 02:05:23.581500 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:23.581478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"]
Jul 29 02:05:24.013651 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:24.013615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd" event={"ID":"f7531c82-0d7d-4ebc-a3c6-5821fd692f04","Type":"ContainerStarted","Data":"97819525244ee7c5d5dfff1ccb4ebbee25d5fc6140e500c459884fe8172c5ce5"}
Jul 29 02:05:24.013798 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:24.013669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd" event={"ID":"f7531c82-0d7d-4ebc-a3c6-5821fd692f04","Type":"ContainerStarted","Data":"6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb"}
Jul 29 02:05:24.031603 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:24.031514    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd" podStartSLOduration=1.031502092 podStartE2EDuration="1.031502092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:05:24.031058704 +0000 UTC m=+3178.462941185" watchObservedRunningTime="2026-07-29 02:05:24.031502092 +0000 UTC m=+3178.463384569"
Jul 29 02:05:25.021533 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:25.021493    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7531c82-0d7d-4ebc-a3c6-5821fd692f04" containerID="97819525244ee7c5d5dfff1ccb4ebbee25d5fc6140e500c459884fe8172c5ce5" exitCode=0
Jul 29 02:05:25.021897 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:25.021580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd" event={"ID":"f7531c82-0d7d-4ebc-a3c6-5821fd692f04","Type":"ContainerDied","Data":"97819525244ee7c5d5dfff1ccb4ebbee25d5fc6140e500c459884fe8172c5ce5"}
Jul 29 02:05:26.155105 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:26.155080    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:26.242022 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:26.241997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca\") pod \"f7531c82-0d7d-4ebc-a3c6-5821fd692f04\" (UID: \"f7531c82-0d7d-4ebc-a3c6-5821fd692f04\") "
Jul 29 02:05:26.242377 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:26.242355    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7531c82-0d7d-4ebc-a3c6-5821fd692f04" (UID: "f7531c82-0d7d-4ebc-a3c6-5821fd692f04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:05:26.343061 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:26.342972    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7531c82-0d7d-4ebc-a3c6-5821fd692f04-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:27.035693 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:27.035663    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"
Jul 29 02:05:27.035864 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:27.035677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd" event={"ID":"f7531c82-0d7d-4ebc-a3c6-5821fd692f04","Type":"ContainerDied","Data":"6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb"}
Jul 29 02:05:27.035864 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:27.035794    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6188b60452c454785914d7e58306b43145f5794caaf1c6e461ee45cd627c4abb"
Jul 29 02:05:28.237621 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.237584    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:05:28.238191 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.237873    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" podUID="d2d5a25e-0d33-4830-a806-3cfee26378e0" containerName="maas-api" containerID="cri-o://4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41" gracePeriod=30
Jul 29 02:05:28.313058 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.313022    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:05:28.313354 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.313297    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" podUID="db534343-679b-411b-8106-a04640b1114b" containerName="payload-pre-processing" containerID="cri-o://651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237" gracePeriod=30
Jul 29 02:05:28.473605 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.473541    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"]
Jul 29 02:05:28.478609 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.477800    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-382dad-l4csd"]
Jul 29 02:05:28.619080 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.619053    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:05:28.622230 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.622210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:05:28.766035 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.765953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6brp9\" (UniqueName: \"kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9\") pod \"db534343-679b-411b-8106-a04640b1114b\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") "
Jul 29 02:05:28.766035 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca\") pod \"d2d5a25e-0d33-4830-a806-3cfee26378e0\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") "
Jul 29 02:05:28.766035 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766031    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume\") pod \"db534343-679b-411b-8106-a04640b1114b\" (UID: \"db534343-679b-411b-8106-a04640b1114b\") "
Jul 29 02:05:28.766345 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdz66\" (UniqueName: \"kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66\") pod \"d2d5a25e-0d33-4830-a806-3cfee26378e0\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") "
Jul 29 02:05:28.766345 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766132    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls\") pod \"d2d5a25e-0d33-4830-a806-3cfee26378e0\" (UID: \"d2d5a25e-0d33-4830-a806-3cfee26378e0\") "
Jul 29 02:05:28.766508 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766474    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2d5a25e-0d33-4830-a806-3cfee26378e0" (UID: "d2d5a25e-0d33-4830-a806-3cfee26378e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:05:28.766568 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766506    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db534343-679b-411b-8106-a04640b1114b" (UID: "db534343-679b-411b-8106-a04640b1114b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:05:28.766625 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.766573    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d5a25e-0d33-4830-a806-3cfee26378e0-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:28.768333 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.768299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9" (OuterVolumeSpecName: "kube-api-access-6brp9") pod "db534343-679b-411b-8106-a04640b1114b" (UID: "db534343-679b-411b-8106-a04640b1114b"). InnerVolumeSpecName "kube-api-access-6brp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:05:28.768747 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.768722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2d5a25e-0d33-4830-a806-3cfee26378e0" (UID: "d2d5a25e-0d33-4830-a806-3cfee26378e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 02:05:28.768836 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.768738    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66" (OuterVolumeSpecName: "kube-api-access-cdz66") pod "d2d5a25e-0d33-4830-a806-3cfee26378e0" (UID: "d2d5a25e-0d33-4830-a806-3cfee26378e0"). InnerVolumeSpecName "kube-api-access-cdz66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:05:28.867139 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.867110    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6brp9\" (UniqueName: \"kubernetes.io/projected/db534343-679b-411b-8106-a04640b1114b-kube-api-access-6brp9\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:28.867247 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.867158    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db534343-679b-411b-8106-a04640b1114b-plugins-config-volume\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:28.867247 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.867174    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdz66\" (UniqueName: \"kubernetes.io/projected/d2d5a25e-0d33-4830-a806-3cfee26378e0-kube-api-access-cdz66\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:28.867247 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:28.867189    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2d5a25e-0d33-4830-a806-3cfee26378e0-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:29.048929 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.048893    2564 generic.go:356] "Generic (PLEG): container finished" podID="db534343-679b-411b-8106-a04640b1114b" containerID="651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237" exitCode=0
Jul 29 02:05:29.049111 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.048938    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"
Jul 29 02:05:29.049111 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.048985    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" event={"ID":"db534343-679b-411b-8106-a04640b1114b","Type":"ContainerDied","Data":"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"}
Jul 29 02:05:29.049111 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.049067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl" event={"ID":"db534343-679b-411b-8106-a04640b1114b","Type":"ContainerDied","Data":"e81c6d0b372a743b8c7b8046cda4d1c5700730ac391a14b1e2b52ac79c4f508a"}
Jul 29 02:05:29.049111 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.049098    2564 scope.go:117] "RemoveContainer" containerID="651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"
Jul 29 02:05:29.050361 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.050337    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2d5a25e-0d33-4830-a806-3cfee26378e0" containerID="4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41" exitCode=0
Jul 29 02:05:29.050472 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.050376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" event={"ID":"d2d5a25e-0d33-4830-a806-3cfee26378e0","Type":"ContainerDied","Data":"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"}
Jul 29 02:05:29.050472 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.050431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt" event={"ID":"d2d5a25e-0d33-4830-a806-3cfee26378e0","Type":"ContainerDied","Data":"51a46bce20ebf877eff2558da53f8f7a7be30ac2de5ef6de24ecefae81f80913"}
Jul 29 02:05:29.050472 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.050386    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"
Jul 29 02:05:29.061034 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.061018    2564 scope.go:117] "RemoveContainer" containerID="651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"
Jul 29 02:05:29.061330 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:05:29.061310    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237\": container with ID starting with 651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237 not found: ID does not exist" containerID="651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"
Jul 29 02:05:29.061402 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.061338    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237"} err="failed to get container status \"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237\": rpc error: code = NotFound desc = could not find container \"651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237\": container with ID starting with 651a5cb043a2fb853dc50c9f30c69e72decfff5f6020cd2c6af97567e0368237 not found: ID does not exist"
Jul 29 02:05:29.061402 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.061356    2564 scope.go:117] "RemoveContainer" containerID="4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"
Jul 29 02:05:29.072005 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.071965    2564 scope.go:117] "RemoveContainer" containerID="4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"
Jul 29 02:05:29.072399 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:05:29.072340    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41\": container with ID starting with 4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41 not found: ID does not exist" containerID="4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"
Jul 29 02:05:29.072399 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.072375    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41"} err="failed to get container status \"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41\": rpc error: code = NotFound desc = could not find container \"4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41\": container with ID starting with 4c67317c581cac2d01b37d2247e1210f2087ca62fcadb4398b846c7f9ebfaf41 not found: ID does not exist"
Jul 29 02:05:29.074778 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.074752    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:05:29.077613 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.077595    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-382dad-864bf9b864-jxhwl"]
Jul 29 02:05:29.088592 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.088573    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:05:29.093646 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:29.093624    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-382dad-5698bf746d-2jrbt"]
Jul 29 02:05:30.117009 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:30.116975    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2d5a25e-0d33-4830-a806-3cfee26378e0" path="/var/lib/kubelet/pods/d2d5a25e-0d33-4830-a806-3cfee26378e0/volumes"
Jul 29 02:05:30.117472 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:30.117451    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db534343-679b-411b-8106-a04640b1114b" path="/var/lib/kubelet/pods/db534343-679b-411b-8106-a04640b1114b/volumes"
Jul 29 02:05:30.117797 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:30.117784    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7531c82-0d7d-4ebc-a3c6-5821fd692f04" path="/var/lib/kubelet/pods/f7531c82-0d7d-4ebc-a3c6-5821fd692f04/volumes"
Jul 29 02:05:38.932451 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932415    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"]
Jul 29 02:05:38.932852 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932838    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d5a25e-0d33-4830-a806-3cfee26378e0" containerName="maas-api"
Jul 29 02:05:38.932904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932853    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d5a25e-0d33-4830-a806-3cfee26378e0" containerName="maas-api"
Jul 29 02:05:38.932904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932869    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db534343-679b-411b-8106-a04640b1114b" containerName="payload-pre-processing"
Jul 29 02:05:38.932904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932875    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db534343-679b-411b-8106-a04640b1114b" containerName="payload-pre-processing"
Jul 29 02:05:38.932904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932883    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7531c82-0d7d-4ebc-a3c6-5821fd692f04" containerName="revoke-keys"
Jul 29 02:05:38.932904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932888    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7531c82-0d7d-4ebc-a3c6-5821fd692f04" containerName="revoke-keys"
Jul 29 02:05:38.933069 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932959    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7531c82-0d7d-4ebc-a3c6-5821fd692f04" containerName="revoke-keys"
Jul 29 02:05:38.933069 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932968    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="db534343-679b-411b-8106-a04640b1114b" containerName="payload-pre-processing"
Jul 29 02:05:38.933069 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.932978    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d5a25e-0d33-4830-a806-3cfee26378e0" containerName="maas-api"
Jul 29 02:05:38.936250 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.936230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:38.938950 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.938925    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:05:38.944039 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.944014    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"]
Jul 29 02:05:38.960594 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:38.960561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv\" (UID: \"edbd5fef-480a-47fd-a0a7-922cbdd48479\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:39.061809 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:39.061778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv\" (UID: \"edbd5fef-480a-47fd-a0a7-922cbdd48479\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:39.062417 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:39.062395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv\" (UID: \"edbd5fef-480a-47fd-a0a7-922cbdd48479\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:39.248709 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:39.248637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:39.389016 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:39.388984    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"]
Jul 29 02:05:39.390741 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:05:39.390715    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbd5fef_480a_47fd_a0a7_922cbdd48479.slice/crio-6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9 WatchSource:0}: Error finding container 6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9: Status 404 returned error can't find the container with id 6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9
Jul 29 02:05:40.125184 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:40.125139    2564 generic.go:356] "Generic (PLEG): container finished" podID="edbd5fef-480a-47fd-a0a7-922cbdd48479" containerID="5bc36d585d97eeee53ba2ad7f2fef6a471cf68f86361fcfbd0aaaa2673106fa5" exitCode=0
Jul 29 02:05:40.125483 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:40.125189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv" event={"ID":"edbd5fef-480a-47fd-a0a7-922cbdd48479","Type":"ContainerDied","Data":"5bc36d585d97eeee53ba2ad7f2fef6a471cf68f86361fcfbd0aaaa2673106fa5"}
Jul 29 02:05:40.125483 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:40.125247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv" event={"ID":"edbd5fef-480a-47fd-a0a7-922cbdd48479","Type":"ContainerStarted","Data":"6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9"}
Jul 29 02:05:41.272042 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:41.272014    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:41.382734 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:41.382654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca\") pod \"edbd5fef-480a-47fd-a0a7-922cbdd48479\" (UID: \"edbd5fef-480a-47fd-a0a7-922cbdd48479\") "
Jul 29 02:05:41.383007 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:41.382979    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "edbd5fef-480a-47fd-a0a7-922cbdd48479" (UID: "edbd5fef-480a-47fd-a0a7-922cbdd48479"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:05:41.483566 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:41.483538    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbd5fef-480a-47fd-a0a7-922cbdd48479-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:42.138997 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:42.138969    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"
Jul 29 02:05:42.139160 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:42.139020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv" event={"ID":"edbd5fef-480a-47fd-a0a7-922cbdd48479","Type":"ContainerDied","Data":"6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9"}
Jul 29 02:05:42.139160 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:42.139075    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6254ed845bc09f0692ad4ff1005cd687900cb047d8413a7dc2af412e389d5ed9"
Jul 29 02:05:54.549029 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.548995    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:05:54.549505 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.549294    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" podUID="072c7f4c-c3cd-414a-8410-93fdba753d8f" containerName="maas-api" containerID="cri-o://369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783" gracePeriod=30
Jul 29 02:05:54.781612 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.781557    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"]
Jul 29 02:05:54.785997 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.785970    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-59f277-q6sgv"]
Jul 29 02:05:54.815907 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.815851    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:05:54.909235 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.909208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dchn\" (UniqueName: \"kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn\") pod \"072c7f4c-c3cd-414a-8410-93fdba753d8f\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") "
Jul 29 02:05:54.909368 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.909285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls\") pod \"072c7f4c-c3cd-414a-8410-93fdba753d8f\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") "
Jul 29 02:05:54.909368 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.909336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca\") pod \"072c7f4c-c3cd-414a-8410-93fdba753d8f\" (UID: \"072c7f4c-c3cd-414a-8410-93fdba753d8f\") "
Jul 29 02:05:54.909741 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.909717    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "072c7f4c-c3cd-414a-8410-93fdba753d8f" (UID: "072c7f4c-c3cd-414a-8410-93fdba753d8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:05:54.911368 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.911345    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "072c7f4c-c3cd-414a-8410-93fdba753d8f" (UID: "072c7f4c-c3cd-414a-8410-93fdba753d8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 02:05:54.911443 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:54.911373    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn" (OuterVolumeSpecName: "kube-api-access-6dchn") pod "072c7f4c-c3cd-414a-8410-93fdba753d8f" (UID: "072c7f4c-c3cd-414a-8410-93fdba753d8f"). InnerVolumeSpecName "kube-api-access-6dchn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:05:55.010779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.010745    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dchn\" (UniqueName: \"kubernetes.io/projected/072c7f4c-c3cd-414a-8410-93fdba753d8f-kube-api-access-6dchn\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:55.010779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.010775    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/072c7f4c-c3cd-414a-8410-93fdba753d8f-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:55.010779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.010785    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/072c7f4c-c3cd-414a-8410-93fdba753d8f-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:05:55.224604 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.224522    2564 generic.go:356] "Generic (PLEG): container finished" podID="072c7f4c-c3cd-414a-8410-93fdba753d8f" containerID="369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783" exitCode=0
Jul 29 02:05:55.224604 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.224564    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"
Jul 29 02:05:55.224802 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.224606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" event={"ID":"072c7f4c-c3cd-414a-8410-93fdba753d8f","Type":"ContainerDied","Data":"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"}
Jul 29 02:05:55.224802 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.224652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l" event={"ID":"072c7f4c-c3cd-414a-8410-93fdba753d8f","Type":"ContainerDied","Data":"5947d290e00d3f14438458810b25d7955abc1005a47707702afe8ce8e6fc0975"}
Jul 29 02:05:55.224802 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.224676    2564 scope.go:117] "RemoveContainer" containerID="369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"
Jul 29 02:05:55.235630 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.235611    2564 scope.go:117] "RemoveContainer" containerID="369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"
Jul 29 02:05:55.235869 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:05:55.235851    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783\": container with ID starting with 369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783 not found: ID does not exist" containerID="369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"
Jul 29 02:05:55.235912 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.235881    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783"} err="failed to get container status \"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783\": rpc error: code = NotFound desc = could not find container \"369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783\": container with ID starting with 369e5a92bc06001413c6b429e4c92b2f5aead0d868c708150df4900a5c4b4783 not found: ID does not exist"
Jul 29 02:05:55.249397 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.249376    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:05:55.255566 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:55.255545    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-59f277-767657f877-z2d5l"]
Jul 29 02:05:56.117708 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:56.117674    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="072c7f4c-c3cd-414a-8410-93fdba753d8f" path="/var/lib/kubelet/pods/072c7f4c-c3cd-414a-8410-93fdba753d8f/volumes"
Jul 29 02:05:56.118071 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:05:56.118017    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edbd5fef-480a-47fd-a0a7-922cbdd48479" path="/var/lib/kubelet/pods/edbd5fef-480a-47fd-a0a7-922cbdd48479/volumes"
Jul 29 02:06:49.794167 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.793220    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"]
Jul 29 02:06:49.794167 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794158    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edbd5fef-480a-47fd-a0a7-922cbdd48479" containerName="revoke-keys"
Jul 29 02:06:49.794759 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794182    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="edbd5fef-480a-47fd-a0a7-922cbdd48479" containerName="revoke-keys"
Jul 29 02:06:49.794759 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794249    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="072c7f4c-c3cd-414a-8410-93fdba753d8f" containerName="maas-api"
Jul 29 02:06:49.794759 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794258    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="072c7f4c-c3cd-414a-8410-93fdba753d8f" containerName="maas-api"
Jul 29 02:06:49.794759 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794451    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="072c7f4c-c3cd-414a-8410-93fdba753d8f" containerName="maas-api"
Jul 29 02:06:49.794759 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.794474    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="edbd5fef-480a-47fd-a0a7-922cbdd48479" containerName="revoke-keys"
Jul 29 02:06:49.798797 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.798765    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:49.802330 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.802293    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:06:49.804859 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.804830    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"]
Jul 29 02:06:49.907225 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:49.907179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s\" (UID: \"3b3c75fe-f754-48f4-a1a2-9567cb125d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:50.007984 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:50.007952    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s\" (UID: \"3b3c75fe-f754-48f4-a1a2-9567cb125d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:50.008696 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:50.008675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s\" (UID: \"3b3c75fe-f754-48f4-a1a2-9567cb125d90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:50.111819 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:50.111724    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:50.258984 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:50.258954    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"]
Jul 29 02:06:50.260883 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:06:50.260854    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3c75fe_f754_48f4_a1a2_9567cb125d90.slice/crio-6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2 WatchSource:0}: Error finding container 6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2: Status 404 returned error can't find the container with id 6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2
Jul 29 02:06:50.577918 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:50.577874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s" event={"ID":"3b3c75fe-f754-48f4-a1a2-9567cb125d90","Type":"ContainerStarted","Data":"6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2"}
Jul 29 02:06:51.585736 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:51.585696    2564 generic.go:356] "Generic (PLEG): container finished" podID="3b3c75fe-f754-48f4-a1a2-9567cb125d90" containerID="7145e860347ed0d2e4249d6a92accb7d744c4422bc4415c75b64cb36a04af2b0" exitCode=0
Jul 29 02:06:51.586221 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:51.585758    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s" event={"ID":"3b3c75fe-f754-48f4-a1a2-9567cb125d90","Type":"ContainerDied","Data":"7145e860347ed0d2e4249d6a92accb7d744c4422bc4415c75b64cb36a04af2b0"}
Jul 29 02:06:52.741317 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:52.741289    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:52.836918 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:52.836881    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca\") pod \"3b3c75fe-f754-48f4-a1a2-9567cb125d90\" (UID: \"3b3c75fe-f754-48f4-a1a2-9567cb125d90\") "
Jul 29 02:06:52.837284 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:52.837260    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b3c75fe-f754-48f4-a1a2-9567cb125d90" (UID: "3b3c75fe-f754-48f4-a1a2-9567cb125d90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:06:52.938142 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:52.938053    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3c75fe-f754-48f4-a1a2-9567cb125d90-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:06:53.599904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:53.599867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s" event={"ID":"3b3c75fe-f754-48f4-a1a2-9567cb125d90","Type":"ContainerDied","Data":"6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2"}
Jul 29 02:06:53.599904 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:53.599902    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"
Jul 29 02:06:53.600114 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:06:53.599923    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d762ba3d125b66b6dcd6025dd5d7ddfbabf42ffda6a213f5fe5dd71688802a2"
Jul 29 02:07:00.449726 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.449687    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:07:00.450984 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.450622    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" podUID="b34f4f25-2978-462b-8137-54fc32321db3" containerName="maas-api" containerID="cri-o://0cabc61af5dd6448601f09d1705a3cc0dcda3157464d18b13a0dc679f69dee2a" gracePeriod=30
Jul 29 02:07:00.649004 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.648960    2564 generic.go:356] "Generic (PLEG): container finished" podID="b34f4f25-2978-462b-8137-54fc32321db3" containerID="0cabc61af5dd6448601f09d1705a3cc0dcda3157464d18b13a0dc679f69dee2a" exitCode=0
Jul 29 02:07:00.649202 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.649031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" event={"ID":"b34f4f25-2978-462b-8137-54fc32321db3","Type":"ContainerDied","Data":"0cabc61af5dd6448601f09d1705a3cc0dcda3157464d18b13a0dc679f69dee2a"}
Jul 29 02:07:00.679497 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.679462    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"]
Jul 29 02:07:00.682090 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.682067    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7f1f73-m6h8s"]
Jul 29 02:07:00.721616 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.721534    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:07:00.811073 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.811040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls\") pod \"b34f4f25-2978-462b-8137-54fc32321db3\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") "
Jul 29 02:07:00.811274 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.811175    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8rkh\" (UniqueName: \"kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh\") pod \"b34f4f25-2978-462b-8137-54fc32321db3\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") "
Jul 29 02:07:00.811274 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.811205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca\") pod \"b34f4f25-2978-462b-8137-54fc32321db3\" (UID: \"b34f4f25-2978-462b-8137-54fc32321db3\") "
Jul 29 02:07:00.811581 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.811555    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b34f4f25-2978-462b-8137-54fc32321db3" (UID: "b34f4f25-2978-462b-8137-54fc32321db3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:07:00.813133 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.813111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b34f4f25-2978-462b-8137-54fc32321db3" (UID: "b34f4f25-2978-462b-8137-54fc32321db3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 02:07:00.813289 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.813265    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh" (OuterVolumeSpecName: "kube-api-access-z8rkh") pod "b34f4f25-2978-462b-8137-54fc32321db3" (UID: "b34f4f25-2978-462b-8137-54fc32321db3"). InnerVolumeSpecName "kube-api-access-z8rkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:07:00.912364 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.912327    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34f4f25-2978-462b-8137-54fc32321db3-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:00.912364 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.912360    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8rkh\" (UniqueName: \"kubernetes.io/projected/b34f4f25-2978-462b-8137-54fc32321db3-kube-api-access-z8rkh\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:00.912622 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:00.912375    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34f4f25-2978-462b-8137-54fc32321db3-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:01.657012 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:01.656979    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"
Jul 29 02:07:01.657508 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:01.657015    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk" event={"ID":"b34f4f25-2978-462b-8137-54fc32321db3","Type":"ContainerDied","Data":"2ba9e880f8b07e9f080384dd87a93d9e04499e44b249d0629025f6887cbf15e2"}
Jul 29 02:07:01.657508 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:01.657081    2564 scope.go:117] "RemoveContainer" containerID="0cabc61af5dd6448601f09d1705a3cc0dcda3157464d18b13a0dc679f69dee2a"
Jul 29 02:07:01.682028 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:01.681996    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:07:01.686413 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:01.686381    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7f1f73-695785c994-9bmrk"]
Jul 29 02:07:02.117847 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:02.117814    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3c75fe-f754-48f4-a1a2-9567cb125d90" path="/var/lib/kubelet/pods/3b3c75fe-f754-48f4-a1a2-9567cb125d90/volumes"
Jul 29 02:07:02.118253 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:02.118238    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34f4f25-2978-462b-8137-54fc32321db3" path="/var/lib/kubelet/pods/b34f4f25-2978-462b-8137-54fc32321db3/volumes"
Jul 29 02:07:07.686813 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.686776    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"]
Jul 29 02:07:07.687277 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687262    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34f4f25-2978-462b-8137-54fc32321db3" containerName="maas-api"
Jul 29 02:07:07.687333 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687278    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34f4f25-2978-462b-8137-54fc32321db3" containerName="maas-api"
Jul 29 02:07:07.687333 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687302    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b3c75fe-f754-48f4-a1a2-9567cb125d90" containerName="revoke-keys"
Jul 29 02:07:07.687333 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687308    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3c75fe-f754-48f4-a1a2-9567cb125d90" containerName="revoke-keys"
Jul 29 02:07:07.687433 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687414    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b3c75fe-f754-48f4-a1a2-9567cb125d90" containerName="revoke-keys"
Jul 29 02:07:07.687433 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.687428    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34f4f25-2978-462b-8137-54fc32321db3" containerName="maas-api"
Jul 29 02:07:07.697534 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.697505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:07.701072 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.701033    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"]
Jul 29 02:07:07.701245 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.701185    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:07:07.776283 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.776248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l\" (UID: \"e6746687-6a46-4caa-9f93-068165f4113d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:07.877203 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.877169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l\" (UID: \"e6746687-6a46-4caa-9f93-068165f4113d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:07.877737 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:07.877717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l\" (UID: \"e6746687-6a46-4caa-9f93-068165f4113d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:08.010447 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:08.010385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:08.151818 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:08.151784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"]
Jul 29 02:07:08.153425 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:07:08.153396    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6746687_6a46_4caa_9f93_068165f4113d.slice/crio-869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35 WatchSource:0}: Error finding container 869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35: Status 404 returned error can't find the container with id 869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35
Jul 29 02:07:08.714571 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:08.714490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l" event={"ID":"e6746687-6a46-4caa-9f93-068165f4113d","Type":"ContainerStarted","Data":"cc4a09d131f372b58eaa9b9bdf996aa1232afe1b035d5688828d75fc21642f4c"}
Jul 29 02:07:08.714571 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:08.714554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l" event={"ID":"e6746687-6a46-4caa-9f93-068165f4113d","Type":"ContainerStarted","Data":"869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35"}
Jul 29 02:07:08.731991 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:08.731949    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l" podStartSLOduration=1.731935488 podStartE2EDuration="1.731935488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:07:08.730335719 +0000 UTC m=+3283.162218200" watchObservedRunningTime="2026-07-29 02:07:08.731935488 +0000 UTC m=+3283.163817968"
Jul 29 02:07:09.723142 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:09.723107    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6746687-6a46-4caa-9f93-068165f4113d" containerID="cc4a09d131f372b58eaa9b9bdf996aa1232afe1b035d5688828d75fc21642f4c" exitCode=0
Jul 29 02:07:09.723512 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:09.723171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l" event={"ID":"e6746687-6a46-4caa-9f93-068165f4113d","Type":"ContainerDied","Data":"cc4a09d131f372b58eaa9b9bdf996aa1232afe1b035d5688828d75fc21642f4c"}
Jul 29 02:07:10.870973 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:10.870951    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:10.902260 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:10.902230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca\") pod \"e6746687-6a46-4caa-9f93-068165f4113d\" (UID: \"e6746687-6a46-4caa-9f93-068165f4113d\") "
Jul 29 02:07:10.902567 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:10.902544    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6746687-6a46-4caa-9f93-068165f4113d" (UID: "e6746687-6a46-4caa-9f93-068165f4113d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:07:11.003103 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:11.003041    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6746687-6a46-4caa-9f93-068165f4113d-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:11.737439 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:11.737411    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"
Jul 29 02:07:11.737609 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:11.737458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l" event={"ID":"e6746687-6a46-4caa-9f93-068165f4113d","Type":"ContainerDied","Data":"869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35"}
Jul 29 02:07:11.737609 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:11.737515    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="869446125c56c7a4318cd11d4b2dba87ff5f1a89b3074bfaa368df6fa5e0bf35"
Jul 29 02:07:18.233424 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.233389    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 02:07:18.233978 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.233624    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" podUID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" containerName="maas-api" containerID="cri-o://f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4" gracePeriod=30
Jul 29 02:07:18.264085 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.264061    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"]
Jul 29 02:07:18.268496 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.268467    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c798cc-29754840-rfk6c"]
Jul 29 02:07:18.453419 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.453388    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"]
Jul 29 02:07:18.456101 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.456078    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c798cc-2ph9l"]
Jul 29 02:07:18.494389 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.494316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 02:07:18.571872 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.571840    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls\") pod \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") "
Jul 29 02:07:18.572052 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.571909    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca\") pod \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") "
Jul 29 02:07:18.572052 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.571966    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blmw7\" (UniqueName: \"kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7\") pod \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\" (UID: \"ffbab41c-bd5c-49d5-9fe8-73c8293543ee\") "
Jul 29 02:07:18.572261 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.572237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffbab41c-bd5c-49d5-9fe8-73c8293543ee" (UID: "ffbab41c-bd5c-49d5-9fe8-73c8293543ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:07:18.574006 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.573983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7" (OuterVolumeSpecName: "kube-api-access-blmw7") pod "ffbab41c-bd5c-49d5-9fe8-73c8293543ee" (UID: "ffbab41c-bd5c-49d5-9fe8-73c8293543ee"). InnerVolumeSpecName "kube-api-access-blmw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:07:18.574134 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.574083    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffbab41c-bd5c-49d5-9fe8-73c8293543ee" (UID: "ffbab41c-bd5c-49d5-9fe8-73c8293543ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 02:07:18.672612 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.672586    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:18.672612 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.672612    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:18.672774 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.672621    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blmw7\" (UniqueName: \"kubernetes.io/projected/ffbab41c-bd5c-49d5-9fe8-73c8293543ee-kube-api-access-blmw7\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:18.782780 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.782748    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" containerID="f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4" exitCode=0
Jul 29 02:07:18.782919 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.782811    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"
Jul 29 02:07:18.782919 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.782841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" event={"ID":"ffbab41c-bd5c-49d5-9fe8-73c8293543ee","Type":"ContainerDied","Data":"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"}
Jul 29 02:07:18.783029 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.782929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w" event={"ID":"ffbab41c-bd5c-49d5-9fe8-73c8293543ee","Type":"ContainerDied","Data":"905b1b8c740f42a14bb462176aaf9dbc39b61e9358145b15b54782c01639a134"}
Jul 29 02:07:18.783029 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.782954    2564 scope.go:117] "RemoveContainer" containerID="f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"
Jul 29 02:07:18.794158 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.794129    2564 scope.go:117] "RemoveContainer" containerID="f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"
Jul 29 02:07:18.794430 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:07:18.794412    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4\": container with ID starting with f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4 not found: ID does not exist" containerID="f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"
Jul 29 02:07:18.794483 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.794438    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4"} err="failed to get container status \"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4\": rpc error: code = NotFound desc = could not find container \"f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4\": container with ID starting with f9ee22260b81327878b26f676cd934d208ce35d913d04d0ad1ee2021220815d4 not found: ID does not exist"
Jul 29 02:07:18.806748 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.806726    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 02:07:18.814466 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:18.814446    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c798cc-5744c85cd8-btl8w"]
Jul 29 02:07:20.116863 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:20.116832    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a23b47bc-90e2-48fa-9d89-5c058baafcdb" path="/var/lib/kubelet/pods/a23b47bc-90e2-48fa-9d89-5c058baafcdb/volumes"
Jul 29 02:07:20.117239 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:20.117124    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6746687-6a46-4caa-9f93-068165f4113d" path="/var/lib/kubelet/pods/e6746687-6a46-4caa-9f93-068165f4113d/volumes"
Jul 29 02:07:20.117523 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:20.117509    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" path="/var/lib/kubelet/pods/ffbab41c-bd5c-49d5-9fe8-73c8293543ee/volumes"
Jul 29 02:07:25.473949 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.473905    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"]
Jul 29 02:07:25.474535 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474513    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6746687-6a46-4caa-9f93-068165f4113d" containerName="revoke-keys"
Jul 29 02:07:25.474620 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474537    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6746687-6a46-4caa-9f93-068165f4113d" containerName="revoke-keys"
Jul 29 02:07:25.474620 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474569    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" containerName="maas-api"
Jul 29 02:07:25.474620 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474578    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" containerName="maas-api"
Jul 29 02:07:25.474779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474707    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffbab41c-bd5c-49d5-9fe8-73c8293543ee" containerName="maas-api"
Jul 29 02:07:25.474779 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.474723    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6746687-6a46-4caa-9f93-068165f4113d" containerName="revoke-keys"
Jul 29 02:07:25.479286 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.479258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:25.482413 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.482388    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chd8\""
Jul 29 02:07:25.485942 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.485907    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"]
Jul 29 02:07:25.533283 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.533261    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq\" (UID: \"2c0a32e1-c77f-4495-b441-510f90a65993\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:25.633965 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.633934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq\" (UID: \"2c0a32e1-c77f-4495-b441-510f90a65993\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:25.634546 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.634529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq\" (UID: \"2c0a32e1-c77f-4495-b441-510f90a65993\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:25.792534 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.792511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:25.933966 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:25.933518    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"]
Jul 29 02:07:26.394858 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.394761    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 02:07:26.409836 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.409819    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 02:07:26.719608 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.719578    2564 scope.go:117] "RemoveContainer" containerID="9733f06084c5532544c5a8e81dc6ef13c968f2c7eb2233fa026274328225a03f"
Jul 29 02:07:26.839686 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.839660    2564 generic.go:356] "Generic (PLEG): container finished" podID="2c0a32e1-c77f-4495-b441-510f90a65993" containerID="e2908616bcb4b687706df518eae05be3ec28c3c51fbee11458e60f2dfd999d4a" exitCode=0
Jul 29 02:07:26.839780 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.839740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq" event={"ID":"2c0a32e1-c77f-4495-b441-510f90a65993","Type":"ContainerDied","Data":"e2908616bcb4b687706df518eae05be3ec28c3c51fbee11458e60f2dfd999d4a"}
Jul 29 02:07:26.839822 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:26.839798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq" event={"ID":"2c0a32e1-c77f-4495-b441-510f90a65993","Type":"ContainerStarted","Data":"eb7094eb50e7c715647d3cd4f156cbf95dc7e128e7613b3d3d9041b6a866a20d"}
Jul 29 02:07:27.971960 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:27.971935    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:28.056361 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.056331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca\") pod \"2c0a32e1-c77f-4495-b441-510f90a65993\" (UID: \"2c0a32e1-c77f-4495-b441-510f90a65993\") "
Jul 29 02:07:28.056714 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.056695    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c0a32e1-c77f-4495-b441-510f90a65993" (UID: "2c0a32e1-c77f-4495-b441-510f90a65993"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:07:28.157312 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.157288    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c0a32e1-c77f-4495-b441-510f90a65993-maas-api-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:28.853467 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.853437    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"
Jul 29 02:07:28.853467 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.853446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq" event={"ID":"2c0a32e1-c77f-4495-b441-510f90a65993","Type":"ContainerDied","Data":"eb7094eb50e7c715647d3cd4f156cbf95dc7e128e7613b3d3d9041b6a866a20d"}
Jul 29 02:07:28.853727 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:28.853497    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb7094eb50e7c715647d3cd4f156cbf95dc7e128e7613b3d3d9041b6a866a20d"
Jul 29 02:07:35.933322 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:35.933286    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 02:07:35.933842 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:35.933561    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" podUID="f3b576e7-c153-453e-95cf-547817cb24d2" containerName="maas-api" containerID="cri-o://f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c" gracePeriod=30
Jul 29 02:07:35.961346 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:35.961319    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"]
Jul 29 02:07:35.964198 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:35.964175    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d32f5-29754840-472z8"]
Jul 29 02:07:36.121321 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.121226    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c3cab41-a318-47fa-a1b3-28bece9dca55" path="/var/lib/kubelet/pods/7c3cab41-a318-47fa-a1b3-28bece9dca55/volumes"
Jul 29 02:07:36.153463 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.153428    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"]
Jul 29 02:07:36.155942 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.155920    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d32f5-nt2rq"]
Jul 29 02:07:36.209249 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.209221    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 02:07:36.325833 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.325796    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") pod \"f3b576e7-c153-453e-95cf-547817cb24d2\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") "
Jul 29 02:07:36.325985 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.325847    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca\") pod \"f3b576e7-c153-453e-95cf-547817cb24d2\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") "
Jul 29 02:07:36.325985 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.325883    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmqhg\" (UniqueName: \"kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg\") pod \"f3b576e7-c153-453e-95cf-547817cb24d2\" (UID: \"f3b576e7-c153-453e-95cf-547817cb24d2\") "
Jul 29 02:07:36.326338 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.326300    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3b576e7-c153-453e-95cf-547817cb24d2" (UID: "f3b576e7-c153-453e-95cf-547817cb24d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 02:07:36.328017 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.327990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg" (OuterVolumeSpecName: "kube-api-access-kmqhg") pod "f3b576e7-c153-453e-95cf-547817cb24d2" (UID: "f3b576e7-c153-453e-95cf-547817cb24d2"). InnerVolumeSpecName "kube-api-access-kmqhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 02:07:36.328314 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.328296    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3b576e7-c153-453e-95cf-547817cb24d2" (UID: "f3b576e7-c153-453e-95cf-547817cb24d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 02:07:36.427134 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.427097    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b576e7-c153-453e-95cf-547817cb24d2-maas-api-tls\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:36.427134 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.427134    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b576e7-c153-453e-95cf-547817cb24d2-openshift-service-ca\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:36.427134 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.427164    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmqhg\" (UniqueName: \"kubernetes.io/projected/f3b576e7-c153-453e-95cf-547817cb24d2-kube-api-access-kmqhg\") on node \"ip-10-0-141-49.ec2.internal\" DevicePath \"\""
Jul 29 02:07:36.905216 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.905179    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3b576e7-c153-453e-95cf-547817cb24d2" containerID="f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c" exitCode=0
Jul 29 02:07:36.905216 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.905215    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"
Jul 29 02:07:36.905439 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.905223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" event={"ID":"f3b576e7-c153-453e-95cf-547817cb24d2","Type":"ContainerDied","Data":"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"}
Jul 29 02:07:36.905439 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.905266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc" event={"ID":"f3b576e7-c153-453e-95cf-547817cb24d2","Type":"ContainerDied","Data":"bb1ae57ec36e5a3bb6ac7912ebeff92d93b67b457b32502e44d6a5e693d5c927"}
Jul 29 02:07:36.905439 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.905286    2564 scope.go:117] "RemoveContainer" containerID="f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"
Jul 29 02:07:36.915709 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.915690    2564 scope.go:117] "RemoveContainer" containerID="f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"
Jul 29 02:07:36.915966 ip-10-0-141-49 kubenswrapper[2564]: E0729 02:07:36.915946    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c\": container with ID starting with f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c not found: ID does not exist" containerID="f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"
Jul 29 02:07:36.916037 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.915974    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c"} err="failed to get container status \"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c\": rpc error: code = NotFound desc = could not find container \"f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c\": container with ID starting with f1a7347c6aced372290c757cf6ee6914af79ac8bcd40481971e5e5a1b777bc6c not found: ID does not exist"
Jul 29 02:07:36.928310 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.928284    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 02:07:36.933842 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:36.933821    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0d32f5-6f555bf7f8-ldqvc"]
Jul 29 02:07:38.117870 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:38.117838    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c0a32e1-c77f-4495-b441-510f90a65993" path="/var/lib/kubelet/pods/2c0a32e1-c77f-4495-b441-510f90a65993/volumes"
Jul 29 02:07:38.118281 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:38.118265    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b576e7-c153-453e-95cf-547817cb24d2" path="/var/lib/kubelet/pods/f3b576e7-c153-453e-95cf-547817cb24d2/volumes"
Jul 29 02:07:43.492682 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:43.492646    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-648f9dbf77-rc9n2_3ce1576e-6034-483c-89b2-9ec23dd55f59/authorino/0.log"
Jul 29 02:07:48.675616 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:48.675538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-trmp8_8f4551ce-4708-4db3-aec3-9b3a3acd75c9/manager/0.log"
Jul 29 02:07:48.785272 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:48.785237    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pzbhb_b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680/manager/0.log"
Jul 29 02:07:49.031245 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.031212    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-742jf_9bb41e47-c95f-4d77-bc73-1480a6eec141/server/0.log"
Jul 29 02:07:49.141522 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.141493    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wwvrd_8eb18c7f-3ea8-4ded-a295-23c4f841faf7/manager/0.log"
Jul 29 02:07:49.244366 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.244333    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f677d9b6b-847s4_063f8783-bc85-45aa-93a9-935de65404af/copy-manifests/0.log"
Jul 29 02:07:49.250221 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.250201    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f677d9b6b-847s4_063f8783-bc85-45aa-93a9-935de65404af/manager/0.log"
Jul 29 02:07:49.824923 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.824884    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74c9c4b7c9-s9vtt_01c33bff-bf76-4e27-8fd2-b1b40dde0b85/maas-api/0.log"
Jul 29 02:07:49.933908 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:49.933882    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754810-s9558_89c80dc1-8bbc-4f4a-89f6-03f0ecdd669f/cleanup/0.log"
Jul 29 02:07:50.042936 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:50.042908    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754825-p92qx_b3f45283-01a4-4967-8d41-19609a891dcf/cleanup/0.log"
Jul 29 02:07:50.145788 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:50.145712    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754840-99lp4_3647fa6b-e15b-48fa-9eb5-58a53c6170b4/cleanup/0.log"
Jul 29 02:07:50.250294 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:50.250271    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qswtb_f2e0be6e-d16d-4fe8-9b16-da7099767911/postgres/0.log"
Jul 29 02:07:51.026663 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.026636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/util/0.log"
Jul 29 02:07:51.032528 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.032501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/pull/0.log"
Jul 29 02:07:51.038090 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.038075    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/extract/0.log"
Jul 29 02:07:51.144365 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.144299    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/pull/0.log"
Jul 29 02:07:51.152212 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.152188    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/extract/0.log"
Jul 29 02:07:51.158512 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.158493    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/util/0.log"
Jul 29 02:07:51.277169 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.277086    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/util/0.log"
Jul 29 02:07:51.288206 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.288186    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/pull/0.log"
Jul 29 02:07:51.304927 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.304888    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/extract/0.log"
Jul 29 02:07:51.427979 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.427953    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/extract/0.log"
Jul 29 02:07:51.436279 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.436256    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/util/0.log"
Jul 29 02:07:51.442526 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.442505    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/pull/0.log"
Jul 29 02:07:51.561910 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.561827    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-648f9dbf77-rc9n2_3ce1576e-6034-483c-89b2-9ec23dd55f59/authorino/0.log"
Jul 29 02:07:51.671923 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.671889    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ppw52_bbe7a322-69fe-4aed-ae5f-8a12f85aa46f/manager/0.log"
Jul 29 02:07:51.787896 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:51.787869    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nkj79_23061558-82b7-4b1f-9de2-e18beb0c4bf8/manager/0.log"
Jul 29 02:07:52.017349 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:52.017280    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k8f6f_a91d336a-4ecf-4483-977e-7eef09289749/registry-server/0.log"
Jul 29 02:07:52.133869 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:52.133841    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hlsrt_bf27adbd-5850-43a5-a34d-bc96fceecd82/manager/0.log"
Jul 29 02:07:52.694056 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:52.694029    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d_46dc861d-630b-4031-9da7-e0a245c3d436/istio-proxy/0.log"
Jul 29 02:07:52.916519 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:52.916492    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dfd7865d-2jsdx_b853d0b7-acd5-4b42-bbd4-86109856d87c/kube-auth-proxy/0.log"
Jul 29 02:07:53.141599 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:53.141569    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/1.log"
Jul 29 02:07:53.484415 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:53.484344    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-857b7c775-7mj7f_e147f4bd-b643-4c77-8ce9-5f7abfe38dec/router/0.log"
Jul 29 02:07:53.814512 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:53.814483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lgwvq_703d8181-4a06-42d3-87d0-7f9d43c018a3/main/0.log"
Jul 29 02:07:54.167445 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:54.167371    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftxlwp_67d4d7f1-e32e-4326-9979-84754e16b460/main/0.log"
Jul 29 02:07:59.880821 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.880788    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8dv4l/must-gather-mxs8l"]
Jul 29 02:07:59.881238 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881223    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b576e7-c153-453e-95cf-547817cb24d2" containerName="maas-api"
Jul 29 02:07:59.881289 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881241    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b576e7-c153-453e-95cf-547817cb24d2" containerName="maas-api"
Jul 29 02:07:59.881289 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881257    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0a32e1-c77f-4495-b441-510f90a65993" containerName="revoke-keys"
Jul 29 02:07:59.881289 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881263    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0a32e1-c77f-4495-b441-510f90a65993" containerName="revoke-keys"
Jul 29 02:07:59.881388 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881343    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b576e7-c153-453e-95cf-547817cb24d2" containerName="maas-api"
Jul 29 02:07:59.881388 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.881352    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0a32e1-c77f-4495-b441-510f90a65993" containerName="revoke-keys"
Jul 29 02:07:59.885960 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.885940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:07:59.888951 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.888926    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dv4l\"/\"openshift-service-ca.crt\""
Jul 29 02:07:59.890232 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.890213    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8dv4l\"/\"kube-root-ca.crt\""
Jul 29 02:07:59.890232 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.890223    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8dv4l\"/\"default-dockercfg-tzxr4\""
Jul 29 02:07:59.897259 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.897235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dv4l/must-gather-mxs8l"]
Jul 29 02:07:59.929522 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.929492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-must-gather-output\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:07:59.929632 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:07:59.929543    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7psj\" (UniqueName: \"kubernetes.io/projected/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-kube-api-access-r7psj\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.030564 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.030533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-must-gather-output\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.030715 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.030576    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7psj\" (UniqueName: \"kubernetes.io/projected/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-kube-api-access-r7psj\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.030870 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.030850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-must-gather-output\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.043448 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.043425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7psj\" (UniqueName: \"kubernetes.io/projected/04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3-kube-api-access-r7psj\") pod \"must-gather-mxs8l\" (UID: \"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3\") " pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.195864 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.195790    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dv4l/must-gather-mxs8l"
Jul 29 02:08:00.341607 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.341578    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dv4l/must-gather-mxs8l"]
Jul 29 02:08:00.343429 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:08:00.343403    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04ac6f2d_b97c_4e7b_8c99_53d07c58d4f3.slice/crio-b7eaca788284733c002ccf8b3e92c1482da2ecf6000e76a5df15740199febbf6 WatchSource:0}: Error finding container b7eaca788284733c002ccf8b3e92c1482da2ecf6000e76a5df15740199febbf6: Status 404 returned error can't find the container with id b7eaca788284733c002ccf8b3e92c1482da2ecf6000e76a5df15740199febbf6
Jul 29 02:08:00.345564 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:00.345543    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 02:08:01.067464 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:01.067427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dv4l/must-gather-mxs8l" event={"ID":"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3","Type":"ContainerStarted","Data":"b7eaca788284733c002ccf8b3e92c1482da2ecf6000e76a5df15740199febbf6"}
Jul 29 02:08:02.080872 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.080838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dv4l/must-gather-mxs8l" event={"ID":"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3","Type":"ContainerStarted","Data":"163d161514e85924cd18ea3baa08196ac868a5990c60d01634cedbccc9601f6e"}
Jul 29 02:08:02.081499 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.081442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dv4l/must-gather-mxs8l" event={"ID":"04ac6f2d-b97c-4e7b-8c99-53d07c58d4f3","Type":"ContainerStarted","Data":"4545e11f7e88db6579ae464a4eb165a2f9a3aee73da9eb9c985c600f57e05330"}
Jul 29 02:08:02.099765 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.099709    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8dv4l/must-gather-mxs8l" podStartSLOduration=2.3572027970000002 podStartE2EDuration="3.09969025s" totalImagesPullingTime="742.487453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 02:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:08:02.094971641 +0000 UTC m=+3336.526854123" watchObservedRunningTime="2026-07-29 02:08:02.09969025 +0000 UTC m=+3336.531572732"
Jul 29 02:08:02.653323 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.653288    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sh9cm_9438ab0c-aec4-41b1-b091-e541c119b72c/global-pull-secret-syncer/0.log"
Jul 29 02:08:02.728625 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.728596    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6kth6_986c3fc6-5c33-494b-8e89-0483e1ddefc7/konnectivity-agent/0.log"
Jul 29 02:08:02.806444 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:02.806407    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-49.ec2.internal_3616ae90a98efffd0c217d8bdda2d07e/haproxy/0.log"
Jul 29 02:08:06.923385 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:06.923352    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/extract/0.log"
Jul 29 02:08:06.946734 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:06.946708    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/util/0.log"
Jul 29 02:08:06.972940 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:06.972908    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fm6ds_0c4f6887-e9a5-4e3e-b5a7-8f6c70ee644b/pull/0.log"
Jul 29 02:08:07.009265 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.009221    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/extract/0.log"
Jul 29 02:08:07.026888 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.026865    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/util/0.log"
Jul 29 02:08:07.047985 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.047960    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hhmx_85b0b85a-588f-4b60-9dac-df9aac75b967/pull/0.log"
Jul 29 02:08:07.086053 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.085939    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/extract/0.log"
Jul 29 02:08:07.104907 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.104825    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/util/0.log"
Jul 29 02:08:07.143060 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.143001    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rw8gv_26f9dc99-fa81-4270-8c7a-6fc877f3fee9/pull/0.log"
Jul 29 02:08:07.171168 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.170826    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/extract/0.log"
Jul 29 02:08:07.194610 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.194581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/util/0.log"
Jul 29 02:08:07.216535 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.216506    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nhwk6_d2274627-6f17-4c41-83fd-355627aa9a46/pull/0.log"
Jul 29 02:08:07.694452 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.694045    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-648f9dbf77-rc9n2_3ce1576e-6034-483c-89b2-9ec23dd55f59/authorino/0.log"
Jul 29 02:08:07.730737 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.730708    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ppw52_bbe7a322-69fe-4aed-ae5f-8a12f85aa46f/manager/0.log"
Jul 29 02:08:07.756171 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.756101    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nkj79_23061558-82b7-4b1f-9de2-e18beb0c4bf8/manager/0.log"
Jul 29 02:08:07.828476 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:07.827529    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k8f6f_a91d336a-4ecf-4483-977e-7eef09289749/registry-server/0.log"
Jul 29 02:08:08.053684 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:08.053582    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hlsrt_bf27adbd-5850-43a5-a34d-bc96fceecd82/manager/0.log"
Jul 29 02:08:10.171014 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.170978    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-qtsqf_198bd2b4-3160-4256-a39f-db5f57868a3f/cluster-monitoring-operator/0.log"
Jul 29 02:08:10.207903 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.207870    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-xdphk_8466b7f3-cff6-43c8-98c7-7fb92ddbf875/kube-state-metrics/0.log"
Jul 29 02:08:10.225801 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.225765    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-xdphk_8466b7f3-cff6-43c8-98c7-7fb92ddbf875/kube-rbac-proxy-main/0.log"
Jul 29 02:08:10.248366 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.248335    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-xdphk_8466b7f3-cff6-43c8-98c7-7fb92ddbf875/kube-rbac-proxy-self/0.log"
Jul 29 02:08:10.421241 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.421167    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zf9_80853a60-2d6a-4dc7-b7bb-10ea0d05af6c/node-exporter/0.log"
Jul 29 02:08:10.444870 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.444837    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zf9_80853a60-2d6a-4dc7-b7bb-10ea0d05af6c/kube-rbac-proxy/0.log"
Jul 29 02:08:10.465719 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:10.465695    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zf9_80853a60-2d6a-4dc7-b7bb-10ea0d05af6c/init-textfile/0.log"
Jul 29 02:08:12.132888 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.132855    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"]
Jul 29 02:08:12.138299 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.138275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.146242 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.146204    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"]
Jul 29 02:08:12.276899 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.276865    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-fljjk_3d35aad0-55d6-4d74-a7d4-fb149201c916/networking-console-plugin/0.log"
Jul 29 02:08:12.284487 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.284460    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-proc\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.284629 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.284510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-lib-modules\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.284629 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.284584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-podres\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.284703 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.284634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjxkx\" (UniqueName: \"kubernetes.io/projected/e24bd953-abc2-4c60-bc7b-a7794525d465-kube-api-access-bjxkx\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.284703 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.284657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-sys\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385317 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-proc\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385317 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-lib-modules\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385562 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-podres\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385562 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjxkx\" (UniqueName: \"kubernetes.io/projected/e24bd953-abc2-4c60-bc7b-a7794525d465-kube-api-access-bjxkx\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385562 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-sys\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385738 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-sys\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385738 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-proc\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385852 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-lib-modules\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.385908 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.385857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e24bd953-abc2-4c60-bc7b-a7794525d465-podres\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.395613 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.395586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjxkx\" (UniqueName: \"kubernetes.io/projected/e24bd953-abc2-4c60-bc7b-a7794525d465-kube-api-access-bjxkx\") pod \"perf-node-gather-daemonset-s2b5x\" (UID: \"e24bd953-abc2-4c60-bc7b-a7794525d465\") " pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.451610 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.451579    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:12.637361 ip-10-0-141-49 kubenswrapper[2564]: W0729 02:08:12.637289    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode24bd953_abc2_4c60_bc7b_a7794525d465.slice/crio-b62eed1da0c4929a0b7cea7f9d66d05a2d91084f3c761d2b4661260479f250ab WatchSource:0}: Error finding container b62eed1da0c4929a0b7cea7f9d66d05a2d91084f3c761d2b4661260479f250ab: Status 404 returned error can't find the container with id b62eed1da0c4929a0b7cea7f9d66d05a2d91084f3c761d2b4661260479f250ab
Jul 29 02:08:12.638662 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.638623    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"]
Jul 29 02:08:12.920617 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.920538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/2.log"
Jul 29 02:08:12.926662 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:12.926639    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-kt2c4_eebd4f30-39fe-4cf3-abf6-d7fd1cc5cff4/console-operator/3.log"
Jul 29 02:08:13.221830 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:13.221737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x" event={"ID":"e24bd953-abc2-4c60-bc7b-a7794525d465","Type":"ContainerStarted","Data":"09eb03aaf7da222e092222cc4f09e1decb48a6d0c0dd491df73b7021f23f2041"}
Jul 29 02:08:13.221830 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:13.221810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x" event={"ID":"e24bd953-abc2-4c60-bc7b-a7794525d465","Type":"ContainerStarted","Data":"b62eed1da0c4929a0b7cea7f9d66d05a2d91084f3c761d2b4661260479f250ab"}
Jul 29 02:08:13.222379 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:13.221958    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:13.242561 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:13.242505    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x" podStartSLOduration=1.242489304 podStartE2EDuration="1.242489304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 02:08:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 02:08:13.24111824 +0000 UTC m=+3347.673000720" watchObservedRunningTime="2026-07-29 02:08:13.242489304 +0000 UTC m=+3347.674371784"
Jul 29 02:08:13.456736 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:13.456687    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67c8645ddf-ghq55_e6f3700e-38d0-441d-895b-c903b5682f60/console/0.log"
Jul 29 02:08:14.033253 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:14.033223    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-djlvl_241109d5-1076-4d7c-bedb-0a1b70487d17/volume-data-source-validator/0.log"
Jul 29 02:08:15.118051 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:15.118023    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lz9dv_95b3711a-bb50-4704-b92e-1bd02d314cd2/dns/0.log"
Jul 29 02:08:15.137530 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:15.137503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lz9dv_95b3711a-bb50-4704-b92e-1bd02d314cd2/kube-rbac-proxy/0.log"
Jul 29 02:08:15.233452 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:15.233419    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tqr8t_1ea819b9-2b2d-4f81-aee1-d6e3636ee41a/dns-node-resolver/0.log"
Jul 29 02:08:15.797685 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:15.797657    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2l955_31ed3c06-7825-4d83-9da3-43877fb9780f/node-ca/0.log"
Jul 29 02:08:16.743303 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:16.743233    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d_46dc861d-630b-4031-9da7-e0a245c3d436/istio-proxy/0.log"
Jul 29 02:08:16.911949 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:16.911920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dfd7865d-2jsdx_b853d0b7-acd5-4b42-bbd4-86109856d87c/kube-auth-proxy/0.log"
Jul 29 02:08:16.962478 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:16.962450    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/0.log"
Jul 29 02:08:16.995303 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:16.995224    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/1.log"
Jul 29 02:08:17.075893 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:17.075864    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-857b7c775-7mj7f_e147f4bd-b643-4c77-8ce9-5f7abfe38dec/router/0.log"
Jul 29 02:08:17.869627 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:17.869602    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-45szv_04414c82-3152-4607-ae22-cae517c84cd4/serve-healthcheck-canary/0.log"
Jul 29 02:08:18.371374 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:18.371340    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wkb2l_49d40c02-c488-48a4-820d-257a0d4ee7b3/insights-operator/0.log"
Jul 29 02:08:18.372306 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:18.372282    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wkb2l_49d40c02-c488-48a4-820d-257a0d4ee7b3/insights-operator/1.log"
Jul 29 02:08:18.521188 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:18.521137    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fbnhs_e3714956-ef11-4e19-810a-0dbefd635a4d/kube-rbac-proxy/0.log"
Jul 29 02:08:18.541774 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:18.541745    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fbnhs_e3714956-ef11-4e19-810a-0dbefd635a4d/exporter/0.log"
Jul 29 02:08:18.560808 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:18.560770    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fbnhs_e3714956-ef11-4e19-810a-0dbefd635a4d/extractor/0.log"
Jul 29 02:08:19.241952 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:19.241924    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8dv4l/perf-node-gather-daemonset-s2b5x"
Jul 29 02:08:20.824984 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:20.824951    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-trmp8_8f4551ce-4708-4db3-aec3-9b3a3acd75c9/manager/0.log"
Jul 29 02:08:21.139999 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:21.139884    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pzbhb_b1d47a29-93d7-4c7a-ba8a-1cb63a3f0680/manager/0.log"
Jul 29 02:08:21.306544 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:21.306508    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-742jf_9bb41e47-c95f-4d77-bc73-1480a6eec141/server/0.log"
Jul 29 02:08:21.404514 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:21.404375    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wwvrd_8eb18c7f-3ea8-4ded-a295-23c4f841faf7/manager/0.log"
Jul 29 02:08:21.454415 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:21.454385    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f677d9b6b-847s4_063f8783-bc85-45aa-93a9-935de65404af/manager/0.log"
Jul 29 02:08:21.490381 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:21.490343    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f677d9b6b-847s4_063f8783-bc85-45aa-93a9-935de65404af/copy-manifests/0.log"
Jul 29 02:08:22.651622 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:22.651585    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d_46dc861d-630b-4031-9da7-e0a245c3d436/istio-proxy/0.log"
Jul 29 02:08:22.795686 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:22.795644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dfd7865d-2jsdx_b853d0b7-acd5-4b42-bbd4-86109856d87c/kube-auth-proxy/0.log"
Jul 29 02:08:22.834276 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:22.834248    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/0.log"
Jul 29 02:08:22.872209 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:22.872174    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/1.log"
Jul 29 02:08:22.924898 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:22.924817    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-857b7c775-7mj7f_e147f4bd-b643-4c77-8ce9-5f7abfe38dec/router/0.log"
Jul 29 02:08:24.274818 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:24.274786    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-4vb6d_46dc861d-630b-4031-9da7-e0a245c3d436/istio-proxy/0.log"
Jul 29 02:08:24.429204 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:24.429090    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dfd7865d-2jsdx_b853d0b7-acd5-4b42-bbd4-86109856d87c/kube-auth-proxy/0.log"
Jul 29 02:08:24.462103 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:24.462076    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/0.log"
Jul 29 02:08:24.497643 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:24.497612    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck9xh_95e13f08-b9f7-4782-84bd-0b62d4a33f76/istio-proxy/1.log"
Jul 29 02:08:24.556906 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:24.556878    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-857b7c775-7mj7f_e147f4bd-b643-4c77-8ce9-5f7abfe38dec/router/0.log"
Jul 29 02:08:26.756629 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:26.756592    2564 scope.go:117] "RemoveContainer" containerID="0a61f2651742081ae68b83cd6861ec84b875170790cd3ee1d87f4f2c5fb31222"
Jul 29 02:08:28.365768 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:28.365724    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl_711d634c-0515-4f70-9f82-6f387aeb020a/kube-storage-version-migrator-operator/1.log"
Jul 29 02:08:28.366981 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:28.366956    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-fd5jl_711d634c-0515-4f70-9f82-6f387aeb020a/kube-storage-version-migrator-operator/0.log"
Jul 29 02:08:29.644631 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:29.644605    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qr45k_6741d76d-6d73-4dad-b5ab-2cb9f3525cfd/kube-multus-additional-cni-plugins/0.log"
Jul 29 02:08:29.664570 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:29.664541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qr45k_6741d76d-6d73-4dad-b5ab-2cb9f3525cfd/egress-router-binary-copy/0.log"
Jul 29 02:08:29.685691 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:29.685667    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qr45k_6741d76d-6d73-4dad-b5ab-2cb9f3525cfd/cni-plugins/0.log"
Jul 29 02:08:29.708883 ip-10-0-141-49 kubenswrapper[2564]: I0729 02:08:29.708864    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qr45k_6741d76d-6d73-4dad-b5ab-2cb9f3525cfd/bond-cni-plugin/0.log"