Jul 28 17:56:58.786588 ip-10-0-133-244 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 17:56:59.192140 ip-10-0-133-244 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:56:59.192140 ip-10-0-133-244 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 17:56:59.192140 ip-10-0-133-244 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:56:59.192140 ip-10-0-133-244 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 17:56:59.192140 ip-10-0-133-244 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 17:56:59.192947 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.192862    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 17:56:59.196279 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196250    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.196279 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196278    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196284    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196287    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196290    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196293    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196296    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196299    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196302    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196305    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196309    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196313    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196316    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196319    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196322    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196324    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196327    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196330    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196332    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196335    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196339    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.196340 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196341    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196344    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196347    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196349    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196352    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196355    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196358    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196362    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196366    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196370    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196372    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196376    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196378    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196381    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196383    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196386    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196388    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196391    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196393    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196396    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.196811 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196398    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196401    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196403    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196406    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196409    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196411    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196414    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196416    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196418    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196421    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196423    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196426    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196428    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196432    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196437    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196440    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196443    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196446    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196449    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.197304 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196454    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196456    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196459    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196461    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196464    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196467    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196470    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196473    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196476    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196478    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196481    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196483    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196486    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196488    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196491    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196493    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196496    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196498    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196501    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.197895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196504    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196507    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196509    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196512    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196514    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196517    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196519    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196522    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196525    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196527    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196530    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196533    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196535    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196538    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196540    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196544    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196546    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196549    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.196551    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197246    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.198658 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197252    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197255    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197278    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197281    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197284    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197287    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197290    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197293    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197295    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197298    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197301    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197305    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197308    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197311    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197314    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197317    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197320    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197322    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197325    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197328    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.199417 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197331    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197333    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197336    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197339    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197341    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197344    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197347    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197349    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197352    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197354    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197357    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197361    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197366    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197369    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197372    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197374    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197377    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197379    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197382    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.199922 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197385    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197387    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197389    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197392    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197396    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197399    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197402    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197404    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197407    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197410    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197412    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197415    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197418    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197422    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197426    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197429    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197431    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197434    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197436    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.200414 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197439    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197442    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197444    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197447    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197450    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197452    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197455    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197457    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197460    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197463    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197465    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197468    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197472    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197476    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197478    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197481    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197484    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197487    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197549    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197601    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.200878 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197606    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197610    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197614    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197617    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197620    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197623    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197627    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197630    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197635    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197637    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197640    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197646    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197649    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197652    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197657    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197661    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197666    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197670    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197675    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.197679    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.201398 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199351    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199364    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199368    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199372    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199375    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199379    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199382    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199385    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199388    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199391    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199393    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199397    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199400    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199403    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199406    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199409    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199412    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199414    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199417    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199420    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.201892 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199423    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199425    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199428    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199431    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199434    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199437    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199439    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199442    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199444    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199447    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199450    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199453    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199456    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199459    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199462    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199465    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199468    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199471    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199473    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199476    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.202413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199479    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199481    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199484    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199487    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199489    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199492    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199494    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199497    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199503    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199505    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199508    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199511    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199513    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199516    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199518    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199521    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199524    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199526    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199529    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199531    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.202914 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199534    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199539    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199543    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199547    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199550    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199553    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199557    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199561    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199564    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199566    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199569    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199573    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199577    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199579    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199582    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199587    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199590    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199593    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.203419 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199596    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199599    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199602    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199604    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199607    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199609    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199612    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199615    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199617    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199620    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199623    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199626    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199628    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199631    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199634    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199637    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199640    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199642    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199645    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199649    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.203916 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.199651    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199722    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199729    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199736    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199740    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199744    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199748    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199752    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199756    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199759    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199763    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199767    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199771    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199774    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199777    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199780    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199783    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199786    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199789    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199793    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199796    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199799    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199802    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199805    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 17:56:59.204509 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199809    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199812    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199815    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199818    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199823    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199826    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199832    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199835    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199838    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199843    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199846    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199849    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199852    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199855    2581 flags.go:64] FLAG: --enable-server="true"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199858    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199862    2581 flags.go:64] FLAG: --event-burst="100"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199866    2581 flags.go:64] FLAG: --event-qps="50"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199868    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199872    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199875    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199878    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199881    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199884    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199888    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199891    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 17:56:59.205084 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199894    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199897    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199900    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199902    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199905    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199909    2581 flags.go:64] FLAG: --feature-gates=""
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199912    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199915    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199918    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199922    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199925    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199928    2581 flags.go:64] FLAG: --help="false"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199932    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199937    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199940    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199943    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199946    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199950    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199953    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199956    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199959    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199962    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199965    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199968    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 17:56:59.205727 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199971    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199974    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199977    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199980    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199983    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199986    2581 flags.go:64] FLAG: --lock-file=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199989    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199992    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.199995    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200001    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200004    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200007    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200010    2581 flags.go:64] FLAG: --logging-format="text"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200013    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200016    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200019    2581 flags.go:64] FLAG: --manifest-url=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200022    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200027    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200030    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200034    2581 flags.go:64] FLAG: --max-pods="110"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200037    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200043    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200046    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200049    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200060    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 17:56:59.206364 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200064    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200067    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200075    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200078    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200081    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200085    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200087    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200093    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200096    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200099    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200102    2581 flags.go:64] FLAG: --port="10250"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200105    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200109    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00446cddd816d897b"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200112    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200115    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200118    2581 flags.go:64] FLAG: --register-node="true"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200120    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200125    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200128    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200130    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200133    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200137    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200140    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200143    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200146    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 17:56:59.206994 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200149    2581 flags.go:64] FLAG: --runonce="false"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200152    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200155    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200158    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200163    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200166    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200169    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200172    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200175    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200178    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200181    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200183    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200186    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200189    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200192    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200195    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200201    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200204    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200208    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200213    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200216    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200218    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200221    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200224    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200227    2581 flags.go:64] FLAG: --v="2"
Jul 28 17:56:59.207622 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200231    2581 flags.go:64] FLAG: --version="false"
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200236    2581 flags.go:64] FLAG: --vmodule=""
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200244    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.200247    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200385    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200389    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200393    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200396    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200399    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200401    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200404    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200408    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200411    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200414    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200417    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200420    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200422    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200425    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200429    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200433    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.208252 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200436    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200439    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200442    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200444    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200447    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200451    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200454    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200457    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200459    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200462    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200465    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200467    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200470    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200472    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200475    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200477    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200480    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200482    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200485    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200487    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.208842 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200490    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200493    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200495    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200499    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200501    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200505    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200507    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200510    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200512    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200515    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200517    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200520    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200522    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200525    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200527    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200530    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200533    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200537    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200540    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200542    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.209383 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200545    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200547    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200550    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200553    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200556    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200558    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200561    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200563    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200566    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200568    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200571    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200573    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200576    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200578    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200581    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200583    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200586    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200588    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200593    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200596    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.209882 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200598    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200601    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200603    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200606    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200608    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200611    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200614    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200617    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200619    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200623    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200626    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200629    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200631    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200634    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200636    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200639    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200642    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200646    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200649    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.210431 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200652    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200656    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200659    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.200662    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.201375    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.207962    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.207980    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.207995    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.208001    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208065    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208071    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208074    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208077    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208080    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.210889 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208083    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208086    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208089    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208092    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208094    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208097    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208099    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208102    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208105    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208107    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208110    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208112    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208115    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208118    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208120    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208123    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208125    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208128    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208131    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208133    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208136    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.211232 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208138    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208141    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208143    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208148    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208151    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208154    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208156    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208159    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208162    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208164    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208167    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208170    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208173    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208175    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208179    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208183    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208186    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208189    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208192    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208196    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.211767 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208198    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208201    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208204    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208207    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208210    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208213    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208216    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208219    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208222    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208225    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208228    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208231    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208234    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208238    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208241    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208244    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208248    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208252    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208255    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.212255 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208270    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208273    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208276    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208279    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208282    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208286    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208289    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208291    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208294    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208297    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208300    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208303    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208306    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208309    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208311    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208314    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208317    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208319    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208322    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.212783 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208324    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208327    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208329    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208332    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208334    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208337    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208340    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208343    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208345    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208348    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208351    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208354    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208358    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208362    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.213234 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208365    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.208370    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208482    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208486    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208489    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208492    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208495    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208498    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208501    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208503    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208506    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208509    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208511    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208514    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208517    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 17:56:59.213593 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208520    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208522    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208525    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208527    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208530    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208532    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208535    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208538    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208540    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208543    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208545    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208548    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208551    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208553    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208556    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208559    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208561    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208564    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208566    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208569    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 17:56:59.213961 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208571    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208574    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208576    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208579    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208581    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208584    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208586    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208589    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208591    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208594    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208596    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208598    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208601    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208604    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208606    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208609    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208613    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208616    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208619    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 17:56:59.214459 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208621    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208624    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208627    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208629    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208632    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208635    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208638    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208640    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208643    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208646    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208649    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208651    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208654    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208656    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208660    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208663    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208666    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208669    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208672    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208675    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 17:56:59.214912 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208678    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208680    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208683    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208686    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208688    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208691    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208693    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208696    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208699    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208702    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208704    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208706    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208709    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208712    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208716    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208719    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208722    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208725    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208728    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208730    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 17:56:59.215413 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208733    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208735    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208738    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208741    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208744    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208746    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:56:59.208749    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.208754    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.208876    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.211097    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.212349    2581 server.go:1019] "Starting client certificate rotation"
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.212446    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.213280    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 17:56:59.215897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.215372    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:56:59.216578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.216565    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 17:56:59.225354 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.225332    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 28 17:56:59.228789 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.228769    2581 log.go:25] "Validated CRI v1 image API"
Jul 28 17:56:59.230752 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.230736    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 17:56:59.233120 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.233099    2581 fs.go:135] Filesystem UUIDs: map[13880c15-ffcf-4dac-818d-b6ee7823c45d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cabcac06-5275-4cc6-9ef7-f26f5e9467d2:/dev/nvme0n1p3]
Jul 28 17:56:59.233194 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.233120    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 17:56:59.238769 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.238654    2581 manager.go:216] Machine: {Timestamp:2026-07-28 17:56:59.237338257 +0000 UTC m=+0.334410520 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099972 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec293d520b543157faf75ab316e7d1de SystemUUID:ec293d52-0b54-3157-faf7-5ab316e7d1de BootID:1951ab01-6163-4fa7-866d-3cfcb933fb54 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:c3:ff:9f:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:c3:ff:9f:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:e6:f4:74:24:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 17:56:59.238769 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.238759    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 17:56:59.238895 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.238843    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 17:56:59.239933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.239916    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:56:59.241437 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.241410    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 17:56:59.241588 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.241439    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 17:56:59.242110 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.242100    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 17:56:59.242140 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.242114    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 17:56:59.242140 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.242122    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:56:59.242759 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.242748    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 17:56:59.243884 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.243874    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 17:56:59.246653 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246642    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:56:59.246778 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246769    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 17:56:59.246818 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246800    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 17:56:59.246848 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246822    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 17:56:59.246848 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246836    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 17:56:59.246919 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246847    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 28 17:56:59.246919 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.246858    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 17:56:59.247205 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.247193    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:56:59.247244 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.247213    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 17:56:59.249546 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.249513    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 17:56:59.250990 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.250976    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 17:56:59.251044 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251007    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 17:56:59.251564 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251546    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 17:56:59.251564 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251566    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251575    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251583    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251595    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251603    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251614    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251622    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251631    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251639    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251650    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 17:56:59.251723 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.251658    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 17:56:59.252482 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.252470    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 17:56:59.252542 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.252485    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 17:56:59.255710 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.255696    2581 server.go:1268] "Started kubelet"
Jul 28 17:56:59.256220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.255831    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 17:56:59.256220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.255821    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 17:56:59.256220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.255932    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 17:56:59.256650 ip-10-0-133-244 systemd[1]: Started Kubernetes Kubelet.
Jul 28 17:56:59.257010 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.256758    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 17:56:59.259885 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.259852    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 17:56:59.259885 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.259855    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 17:56:59.260744 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.260725    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 17:56:59.261625 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.261607    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 17:56:59.262276 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.262241    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 17:56:59.265976 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.265958    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pl25p"
Jul 28 17:56:59.267070 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.266229    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-244.ec2.internal.18c685cd55f178bc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-244.ec2.internal,UID:ip-10-0-133-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-244.ec2.internal,},FirstTimestamp:2026-07-28 17:56:59.255658684 +0000 UTC m=+0.352730945,LastTimestamp:2026-07-28 17:56:59.255658684 +0000 UTC m=+0.352730945,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-244.ec2.internal,}"
Jul 28 17:56:59.267935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.267914    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 17:56:59.268009 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.267932    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 17:56:59.268072 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.268012    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.268749 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.268728    2581 factory.go:55] Registering systemd factory
Jul 28 17:56:59.268749 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.268752    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 28 17:56:59.269003 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.268987    2581 factory.go:153] Registering CRI-O factory
Jul 28 17:56:59.269092 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269076    2581 factory.go:223] Registration of the crio container factory successfully
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269125    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269044    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269044    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269148    2581 factory.go:103] Registering Raw factory
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269152    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 17:56:59.269193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269163    2581 manager.go:1196] Started watching for new ooms in manager
Jul 28 17:56:59.269477 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269283    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 17:56:59.269477 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269292    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 17:56:59.269577 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.269503    2581 manager.go:349] Starting recovery of all containers
Jul 28 17:56:59.274932 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.274882    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pl25p"
Jul 28 17:56:59.275485 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.275428    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 17:56:59.275575 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.275545    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 17:56:59.281032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.281015    2581 manager.go:354] Recovery completed
Jul 28 17:56:59.285857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.285844    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.287703 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.287681    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.287776 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.287723    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.287776 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.287739    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.290135 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290113    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 17:56:59.290135 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290128    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 17:56:59.290292 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290148    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 17:56:59.290292 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290164    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.290706 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.290628    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-244.ec2.internal.18c685cd57da6b8c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-244.ec2.internal,UID:ip-10-0-133-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-244.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-244.ec2.internal,},FirstTimestamp:2026-07-28 17:56:59.287702412 +0000 UTC m=+0.384774670,LastTimestamp:2026-07-28 17:56:59.287702412 +0000 UTC m=+0.384774670,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-244.ec2.internal,}"
Jul 28 17:56:59.290902 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290888    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.290950 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290910    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.290950 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.290920    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.291320 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.291307    2581 policy_none.go:49] "None policy: Start"
Jul 28 17:56:59.291391 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.291325    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 17:56:59.291391 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.291338    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 17:56:59.292303 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.292290    2581 policy_none.go:47] "Start"
Jul 28 17:56:59.324854 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.324838    2581 manager.go:337] "Starting Device Plugin manager"
Jul 28 17:56:59.324941 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.324871    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 17:56:59.324941 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.324881    2581 server.go:87] "Starting device plugin registration server"
Jul 28 17:56:59.325149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.325136    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 17:56:59.325202 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.325152    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 17:56:59.325275 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.325245    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 17:56:59.325367 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.325345    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 17:56:59.325367 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.325357    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 17:56:59.325847 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.325814    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 17:56:59.325942 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.325869    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.426307 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.426274    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.427365 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.427348    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.427442 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.427382    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.427442 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.427395    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.427442 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.427423    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.429575 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.429544    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 17:56:59.430751 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.430725    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 17:56:59.430834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.430771    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 17:56:59.430834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.430798    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 17:56:59.430935 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.430837    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 17:56:59.433338 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.433322    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:56:59.433725 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.433711    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.433768 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.433740    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-244.ec2.internal\": node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.476871 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.476815    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.531378 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.531336    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"]
Jul 28 17:56:59.531554 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.531416    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.533172 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.533147    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.533305 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.533189    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.533305 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.533203    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.534612 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.534599    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.534745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.534729    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.534786 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.534761    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.536043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536028    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.536163 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536060    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.536163 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536070    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.536163 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536030    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.536163 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536126    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.536163 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.536150    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.537518 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.537502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.537559 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.537530    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 17:56:59.538403 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.538385    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 17:56:59.538480 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.538414    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 17:56:59.538480 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.538424    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeHasSufficientPID"
Jul 28 17:56:59.554811 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.554792    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-244.ec2.internal\" not found" node="ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.558862 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.558846    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-244.ec2.internal\" not found" node="ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.571097 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.571065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/505a36e67b1ce4359d0b61ae9f5542fa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-244.ec2.internal\" (UID: \"505a36e67b1ce4359d0b61ae9f5542fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.571178 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.571106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.571178 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.571125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.577786 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.577768    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.671522 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.671522 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/505a36e67b1ce4359d0b61ae9f5542fa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-244.ec2.internal\" (UID: \"505a36e67b1ce4359d0b61ae9f5542fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.671701 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.671701 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671597    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.671701 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7312aca751091fa0f1482dbad6f5763c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal\" (UID: \"7312aca751091fa0f1482dbad6f5763c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.671701 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.671596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/505a36e67b1ce4359d0b61ae9f5542fa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-244.ec2.internal\" (UID: \"505a36e67b1ce4359d0b61ae9f5542fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.678571 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.678553    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.779700 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.779614    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.856843 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.856815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.862335 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:56:59.862134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:56:59.880713 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.880669    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:56:59.981296 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:56:59.981247    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:57:01.608876 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:01.608841    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:57:01.709491 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:01.709461    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:57:01.738670 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.738635    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 17:57:01.739130 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.738803    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 17:57:01.809760 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:01.809722    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:57:01.810822 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.810797    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:52:00 +0000 UTC" deadline="2028-03-14 10:33:09.157815888 +0000 UTC"
Jul 28 17:57:01.810881 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.810827    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14272h36m7.34699704s"
Jul 28 17:57:01.910545 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:01.910514    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-244.ec2.internal\" not found"
Jul 28 17:57:01.938971 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.938943    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:01.967675 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:01.967643    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod505a36e67b1ce4359d0b61ae9f5542fa.slice/crio-a3e6e81834fddcd6c124a540bcaed9147dce0adae72e11e065dcced66a6825be WatchSource:0}: Error finding container a3e6e81834fddcd6c124a540bcaed9147dce0adae72e11e065dcced66a6825be: Status 404 returned error can't find the container with id a3e6e81834fddcd6c124a540bcaed9147dce0adae72e11e065dcced66a6825be
Jul 28 17:57:01.967841 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:01.967828    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7312aca751091fa0f1482dbad6f5763c.slice/crio-feabdafc4c189f993b6c4c8a84038b683d26960cec57b4706d9da35bcb7888a7 WatchSource:0}: Error finding container feabdafc4c189f993b6c4c8a84038b683d26960cec57b4706d9da35bcb7888a7: Status 404 returned error can't find the container with id feabdafc4c189f993b6c4c8a84038b683d26960cec57b4706d9da35bcb7888a7
Jul 28 17:57:01.975570 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.975549    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 17:57:01.996069 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:01.996040    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal"
Jul 28 17:57:02.007156 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.007135    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:57:02.008964 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.008948    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal"
Jul 28 17:57:02.019215 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.018004    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 17:57:02.157301 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.157201    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:02.319500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.319473    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:02.603036 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.603003    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 17:57:02.775445 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.775412    2581 apiserver.go:52] "Watching apiserver"
Jul 28 17:57:02.781537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.781507    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 17:57:02.782532 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.782497    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-kc972","openshift-image-registry/node-ca-sl47n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal","openshift-multus/multus-2shpt","openshift-network-operator/iptables-alerter-s8m4k","kube-system/global-pull-secret-syncer-8lwbk","kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal","openshift-cluster-node-tuning-operator/tuned-2c7f8","openshift-multus/multus-additional-cni-plugins-jrf4c","openshift-multus/network-metrics-daemon-rqhrw","openshift-network-diagnostics/network-check-target-tsqvb","openshift-ovn-kubernetes/ovnkube-node-j9lbg","kube-system/konnectivity-agent-m9m2v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"]
Jul 28 17:57:02.784068 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.784048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.785837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.785169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.786405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.786371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:02.786485 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.786407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.786485 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.786453    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:02.786737 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.786722    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.786852 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.786834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mgxst\""
Jul 28 17:57:02.787391 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.787372    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.787602 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.787580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 17:57:02.788094 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.788064    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.788181 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.788159    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:02.788541 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.788523    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rdxdr\""
Jul 28 17:57:02.789069 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.788818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.789228 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.789208    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.789805 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.789791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.789886 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.789849    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:02.790901 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.790868    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.791019 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.790999    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qpx4p\""
Jul 28 17:57:02.791164 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.791147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.791365 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.791325    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 17:57:02.791684 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.791665    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.792611 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.792591    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.793243 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.793133    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.794195 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.793887    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pjqx7\""
Jul 28 17:57:02.794195 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.794045    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.795668 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.795648    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.795668 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.795659    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 17:57:02.796283 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.796248    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.796394 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.796378    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 17:57:02.796439 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.796392    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 17:57:02.796704 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.796692    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 17:57:02.796910 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.796894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.797174 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.797157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.798004 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.797983    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fd8pz\""
Jul 28 17:57:02.800586 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.800562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:02.801116 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 17:57:02.801429 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801394    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 17:57:02.801531 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xtwq9\""
Jul 28 17:57:02.801730 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801594    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 17:57:02.801730 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801649    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.801861 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801812    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.801946 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.801931    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fd2r7\""
Jul 28 17:57:02.802519 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.802496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 17:57:02.802623 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.802530    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 17:57:02.802816 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.802796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 17:57:02.802903 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.802858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qcstf\""
Jul 28 17:57:02.802903 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.802896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:02.803011 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.803000    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 17:57:02.805437 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.805409    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 17:57:02.805587 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.805480    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 17:57:02.805701 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.805682    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-99zv4\""
Jul 28 17:57:02.806060 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 17:57:02.806827 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.806944 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-netns\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.806944 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-tuned\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.806944 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-k8s-cni-cncf-io\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.807090 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.807090 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.806979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-registration-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.807090 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-kubernetes\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.807090 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-lib-modules\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.807090 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-system-cni-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-hostroot\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-modprobe-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-bin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-systemd-units\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-node-log\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-netd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td56g\" (UniqueName: \"kubernetes.io/projected/1f621ff1-7734-4d8b-b424-583b35762678-kube-api-access-td56g\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-sys\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cnibin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-kubelet\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-etc-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.807568 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-ovn\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-env-overrides\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25n2b\" (UniqueName: \"kubernetes.io/projected/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-kube-api-access-25n2b\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4tbd\" (UniqueName: \"kubernetes.io/projected/9e960f89-a33d-4495-b827-70de2eddc0ab-kube-api-access-t4tbd\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysconfig\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-systemd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-var-lib-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-config\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f621ff1-7734-4d8b-b424-583b35762678-iptables-alerter-script\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-dbus\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-conf\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac26091-4d06-4f7a-bcf7-acde7784cc74-hosts-file\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac26091-4d06-4f7a-bcf7-acde7784cc74-tmp-dir\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-kubelet-config\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl472\" (UniqueName: \"kubernetes.io/projected/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-kube-api-access-bl472\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.808017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-netns\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.807996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-multus\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-device-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-sys-fs\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-systemd\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-var-lib-kubelet\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-bin\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f621ff1-7734-4d8b-b424-583b35762678-host-slash\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v574\" (UniqueName: \"kubernetes.io/projected/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-kube-api-access-5v574\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-system-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-os-release\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-log-socket\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.808774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808331    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-socket-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cnibin\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-kubelet\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgg9h\" (UniqueName: \"kubernetes.io/projected/7ac26091-4d06-4f7a-bcf7-acde7784cc74-kube-api-access-tgg9h\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-run\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-tmp\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-socket-dir-parent\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-os-release\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cni-binary-copy\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-script-lib\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-host\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-conf-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.809715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-daemon-config\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.810578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-multus-certs\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.810578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-etc-kubernetes\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.810578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-slash\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.810578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4xh\" (UniqueName: \"kubernetes.io/projected/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-kube-api-access-fw4xh\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.810578 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.808865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwpwt\" (UniqueName: \"kubernetes.io/projected/5dc72e56-2e79-423c-aca1-db45ca996cda-kube-api-access-hwpwt\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.810919 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.810837    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 17:57:02.834979 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.834951    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-64wck"
Jul 28 17:57:02.842424 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.842399    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-64wck"
Jul 28 17:57:02.897703 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.897674    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 17:57:02.909119 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e67f7db-91a9-4e7f-bfcc-21b79228466d-host\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e67f7db-91a9-4e7f-bfcc-21b79228466d-serviceca\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-socket-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cnibin\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-kubelet\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgg9h\" (UniqueName: \"kubernetes.io/projected/7ac26091-4d06-4f7a-bcf7-acde7784cc74-kube-api-access-tgg9h\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-socket-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-kubelet\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cnibin\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-run\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-tmp\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-socket-dir-parent\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-os-release\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-run\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cni-binary-copy\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-socket-dir-parent\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.909705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-script-lib\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-host\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909746    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-host\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909789    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-os-release\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-conf-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-conf-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-daemon-config\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-multus-certs\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.909872    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-etc-kubernetes\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-slash\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.909943    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:03.409923869 +0000 UTC m=+2.979986718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909957    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-slash\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4xh\" (UniqueName: \"kubernetes.io/projected/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-kube-api-access-fw4xh\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-multus-certs\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.909998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwpwt\" (UniqueName: \"kubernetes.io/projected/5dc72e56-2e79-423c-aca1-db45ca996cda-kube-api-access-hwpwt\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.910500 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cni-binary-copy\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910202    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-etc-kubernetes\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-netns\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-tuned\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-k8s-cni-cncf-io\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-netns\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-run-k8s-cni-cncf-io\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-registration-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-kubernetes\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-lib-modules\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-system-cni-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-hostroot\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-registration-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.911412 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-modprobe-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-bin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.910862    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-systemd-units\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.910924    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:03.410904608 +0000 UTC m=+2.980967442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-systemd-units\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-node-log\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-daemon-config\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-netd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.910999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-kubernetes\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-script-lib\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15fec211-1790-4736-a469-069077d4c7fa-agent-certs\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-hostroot\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td56g\" (UniqueName: \"kubernetes.io/projected/1f621ff1-7734-4d8b-b424-583b35762678-kube-api-access-td56g\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911057    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-sys\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-system-cni-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.912196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-modprobe-d\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911046    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-bin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-node-log\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cnibin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-sys\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-lib-modules\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-kubelet\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-etc-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-netd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-ovn\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-env-overrides\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25n2b\" (UniqueName: \"kubernetes.io/projected/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-kube-api-access-25n2b\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnrj\" (UniqueName: \"kubernetes.io/projected/2e67f7db-91a9-4e7f-bfcc-21b79228466d-kube-api-access-zbnrj\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-cnibin\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.913099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4tbd\" (UniqueName: \"kubernetes.io/projected/9e960f89-a33d-4495-b827-70de2eddc0ab-kube-api-access-t4tbd\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysconfig\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-ovn\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-systemd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-var-lib-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-config\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f621ff1-7734-4d8b-b424-583b35762678-iptables-alerter-script\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-dbus\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-conf\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911765    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15fec211-1790-4736-a469-069077d4c7fa-konnectivity-ca\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-dbus\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac26091-4d06-4f7a-bcf7-acde7784cc74-hosts-file\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-run-systemd\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911823    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac26091-4d06-4f7a-bcf7-acde7784cc74-tmp-dir\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-var-lib-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.913894 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-kubelet-config\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl472\" (UniqueName: \"kubernetes.io/projected/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-kube-api-access-bl472\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911945    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysconfig\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-env-overrides\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.911993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-netns\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-run-netns\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac26091-4d06-4f7a-bcf7-acde7784cc74-hosts-file\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac26091-4d06-4f7a-bcf7-acde7784cc74-tmp-dir\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovnkube-config\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-sysctl-conf\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-kubelet\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-etc-openvswitch\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-kubelet-config\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-multus\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-device-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-sys-fs\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.914691 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-systemd\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912679    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-var-lib-kubelet\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-systemd\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-var-lib-kubelet\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-device-dir\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e960f89-a33d-4495-b827-70de2eddc0ab-sys-fs\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-host-var-lib-cni-multus\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.912978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-bin\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f621ff1-7734-4d8b-b424-583b35762678-host-slash\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v574\" (UniqueName: \"kubernetes.io/projected/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-kube-api-access-5v574\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-system-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1f621ff1-7734-4d8b-b424-583b35762678-iptables-alerter-script\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-os-release\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-host-cni-bin\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.915435 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-log-socket\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.913238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1f621ff1-7734-4d8b-b424-583b35762678-host-slash\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.914451    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-multus-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.914607    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.914659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-os-release\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.915483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-etc-tuned\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.915916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-ovn-node-metrics-cert\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.916012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-system-cni-dir\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.916197 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.916070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-log-socket\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.918601 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.918474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-tmp\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.919900 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.919879    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:02.920014 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.919904    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:02.920014 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.919920    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:02.920148 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:02.920044    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:03.420016749 +0000 UTC m=+2.990079573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:02.921011 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.920655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgg9h\" (UniqueName: \"kubernetes.io/projected/7ac26091-4d06-4f7a-bcf7-acde7784cc74-kube-api-access-tgg9h\") pod \"node-resolver-kc972\" (UID: \"7ac26091-4d06-4f7a-bcf7-acde7784cc74\") " pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:02.921714 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.921691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwpwt\" (UniqueName: \"kubernetes.io/projected/5dc72e56-2e79-423c-aca1-db45ca996cda-kube-api-access-hwpwt\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:02.922884 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.922844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25n2b\" (UniqueName: \"kubernetes.io/projected/b3373d24-6d72-4a4b-8be6-bb3604dd1c5a-kube-api-access-25n2b\") pod \"ovnkube-node-j9lbg\" (UID: \"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:02.923023 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.923001    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td56g\" (UniqueName: \"kubernetes.io/projected/1f621ff1-7734-4d8b-b424-583b35762678-kube-api-access-td56g\") pod \"iptables-alerter-s8m4k\" (UID: \"1f621ff1-7734-4d8b-b424-583b35762678\") " pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:02.923113 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.923089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4xh\" (UniqueName: \"kubernetes.io/projected/6a4b1fd7-dad0-4eb2-88dc-bf34529844e7-kube-api-access-fw4xh\") pod \"multus-additional-cni-plugins-jrf4c\" (UID: \"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7\") " pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:02.923402 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.923378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl472\" (UniqueName: \"kubernetes.io/projected/2ed0a3f3-869e-4539-87b3-9a7cbda2f78e-kube-api-access-bl472\") pod \"multus-2shpt\" (UID: \"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e\") " pod="openshift-multus/multus-2shpt"
Jul 28 17:57:02.923905 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.923887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4tbd\" (UniqueName: \"kubernetes.io/projected/9e960f89-a33d-4495-b827-70de2eddc0ab-kube-api-access-t4tbd\") pod \"aws-ebs-csi-driver-node-9lpmb\" (UID: \"9e960f89-a33d-4495-b827-70de2eddc0ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:02.927375 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.927357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v574\" (UniqueName: \"kubernetes.io/projected/bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94-kube-api-access-5v574\") pod \"tuned-2c7f8\" (UID: \"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94\") " pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:02.962609 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.962555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal" event={"ID":"505a36e67b1ce4359d0b61ae9f5542fa","Type":"ContainerStarted","Data":"a3e6e81834fddcd6c124a540bcaed9147dce0adae72e11e065dcced66a6825be"}
Jul 28 17:57:02.963698 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:02.963671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal" event={"ID":"7312aca751091fa0f1482dbad6f5763c","Type":"ContainerStarted","Data":"feabdafc4c189f993b6c4c8a84038b683d26960cec57b4706d9da35bcb7888a7"}
Jul 28 17:57:03.014458 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15fec211-1790-4736-a469-069077d4c7fa-agent-certs\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:03.014643 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnrj\" (UniqueName: \"kubernetes.io/projected/2e67f7db-91a9-4e7f-bfcc-21b79228466d-kube-api-access-zbnrj\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.014643 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15fec211-1790-4736-a469-069077d4c7fa-konnectivity-ca\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:03.014643 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e67f7db-91a9-4e7f-bfcc-21b79228466d-host\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.014643 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e67f7db-91a9-4e7f-bfcc-21b79228466d-serviceca\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.014862 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.014636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e67f7db-91a9-4e7f-bfcc-21b79228466d-host\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.015075 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.015049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e67f7db-91a9-4e7f-bfcc-21b79228466d-serviceca\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.015211 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.015165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15fec211-1790-4736-a469-069077d4c7fa-konnectivity-ca\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:03.017224 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.017196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15fec211-1790-4736-a469-069077d4c7fa-agent-certs\") pod \"konnectivity-agent-m9m2v\" (UID: \"15fec211-1790-4736-a469-069077d4c7fa\") " pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:03.022843 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.022823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnrj\" (UniqueName: \"kubernetes.io/projected/2e67f7db-91a9-4e7f-bfcc-21b79228466d-kube-api-access-zbnrj\") pod \"node-ca-sl47n\" (UID: \"2e67f7db-91a9-4e7f-bfcc-21b79228466d\") " pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.097522 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.097483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kc972"
Jul 28 17:57:03.106001 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.105968    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb"
Jul 28 17:57:03.117795 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.117721    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8m4k"
Jul 28 17:57:03.124505 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.124481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2c7f8"
Jul 28 17:57:03.132162 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.132136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jrf4c"
Jul 28 17:57:03.139863 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.139838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:03.147689 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.147669    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2shpt"
Jul 28 17:57:03.155363 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.155339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:03.162220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.162197    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sl47n"
Jul 28 17:57:03.417543 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.417452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:03.417543 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.417495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:03.417740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.417606    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:03.417740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.417667    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:04.417653117 +0000 UTC m=+3.987715929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:03.417740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.417606    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:03.417740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.417738    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:04.417727235 +0000 UTC m=+3.987790044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:03.518340 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.518295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:03.518538 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.518469    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:03.518538 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.518491    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:03.518538 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.518504    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:03.518696 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.518606    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:04.518586965 +0000 UTC m=+4.088649779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:03.845195 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.845153    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:52:02 +0000 UTC" deadline="2028-02-11 18:11:13.40359082 +0000 UTC"
Jul 28 17:57:03.845195 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.845191    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13512h14m9.558404128s"
Jul 28 17:57:03.864701 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:03.864676    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0b3a5a_e281_410c_a2f8_a1c8e1fc6e94.slice/crio-947fdf6022834b63b651e68ae9e25536d04b69b41e7dfcac1b0779111791b127 WatchSource:0}: Error finding container 947fdf6022834b63b651e68ae9e25536d04b69b41e7dfcac1b0779111791b127: Status 404 returned error can't find the container with id 947fdf6022834b63b651e68ae9e25536d04b69b41e7dfcac1b0779111791b127
Jul 28 17:57:03.877492 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:03.877468    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed0a3f3_869e_4539_87b3_9a7cbda2f78e.slice/crio-1709d2298de1a825409587d42e12ea32d78a523e2e2cda0cefcc4c85f44bbb3f WatchSource:0}: Error finding container 1709d2298de1a825409587d42e12ea32d78a523e2e2cda0cefcc4c85f44bbb3f: Status 404 returned error can't find the container with id 1709d2298de1a825409587d42e12ea32d78a523e2e2cda0cefcc4c85f44bbb3f
Jul 28 17:57:03.878910 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:03.878877    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e960f89_a33d_4495_b827_70de2eddc0ab.slice/crio-368359de1cad160dd07ae3242b210c54f0c7b5f455d68395d8e74bb104c71d54 WatchSource:0}: Error finding container 368359de1cad160dd07ae3242b210c54f0c7b5f455d68395d8e74bb104c71d54: Status 404 returned error can't find the container with id 368359de1cad160dd07ae3242b210c54f0c7b5f455d68395d8e74bb104c71d54
Jul 28 17:57:03.881895 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:03.881871    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fec211_1790_4736_a469_069077d4c7fa.slice/crio-00de78b6c38fa638dbc4cc5e41f2509597eb5dc29f3b9cf0f53d0436a81cf6f2 WatchSource:0}: Error finding container 00de78b6c38fa638dbc4cc5e41f2509597eb5dc29f3b9cf0f53d0436a81cf6f2: Status 404 returned error can't find the container with id 00de78b6c38fa638dbc4cc5e41f2509597eb5dc29f3b9cf0f53d0436a81cf6f2
Jul 28 17:57:03.883522 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:03.883489    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4b1fd7_dad0_4eb2_88dc_bf34529844e7.slice/crio-184a562675833b29db321f2eecc776d0568a4755fd578d9badfec31e17efaba1 WatchSource:0}: Error finding container 184a562675833b29db321f2eecc776d0568a4755fd578d9badfec31e17efaba1: Status 404 returned error can't find the container with id 184a562675833b29db321f2eecc776d0568a4755fd578d9badfec31e17efaba1
Jul 28 17:57:03.959006 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.958981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:03.959137 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.959005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:03.959137 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.959109    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:03.959295 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:03.959258    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:03.967113 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.967085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerStarted","Data":"184a562675833b29db321f2eecc776d0568a4755fd578d9badfec31e17efaba1"}
Jul 28 17:57:03.968667 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.968619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m9m2v" event={"ID":"15fec211-1790-4736-a469-069077d4c7fa","Type":"ContainerStarted","Data":"00de78b6c38fa638dbc4cc5e41f2509597eb5dc29f3b9cf0f53d0436a81cf6f2"}
Jul 28 17:57:03.974253 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.974224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sl47n" event={"ID":"2e67f7db-91a9-4e7f-bfcc-21b79228466d","Type":"ContainerStarted","Data":"61ec99f13463a719d5b998b9b3fae2a54c525f55992710a43a01c37be224ba8d"}
Jul 28 17:57:03.976021 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.975967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kc972" event={"ID":"7ac26091-4d06-4f7a-bcf7-acde7784cc74","Type":"ContainerStarted","Data":"c0440b44b60f9c3bb4ff662d854ff9db9d784a13cda53da82d053c3cb1829364"}
Jul 28 17:57:03.977189 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.977170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb" event={"ID":"9e960f89-a33d-4495-b827-70de2eddc0ab","Type":"ContainerStarted","Data":"368359de1cad160dd07ae3242b210c54f0c7b5f455d68395d8e74bb104c71d54"}
Jul 28 17:57:03.978381 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.978357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2shpt" event={"ID":"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e","Type":"ContainerStarted","Data":"1709d2298de1a825409587d42e12ea32d78a523e2e2cda0cefcc4c85f44bbb3f"}
Jul 28 17:57:03.979440 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.979415    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2c7f8" event={"ID":"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94","Type":"ContainerStarted","Data":"947fdf6022834b63b651e68ae9e25536d04b69b41e7dfcac1b0779111791b127"}
Jul 28 17:57:03.980466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.980432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8m4k" event={"ID":"1f621ff1-7734-4d8b-b424-583b35762678","Type":"ContainerStarted","Data":"9877b42926463762985e7a307965c5e3f3680ded52faa01c459d5745b9810caf"}
Jul 28 17:57:03.981481 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:03.981457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"6cd26dee3138a097022f48aa8652d028712d4cddac9b1dc7669098bed4efd612"}
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.425264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.425589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.425533    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.425773    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:06.425756566 +0000 UTC m=+5.995819381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.425713    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:04.426237 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.426151    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:06.426128061 +0000 UTC m=+5.996190875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:04.527302 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.526579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:04.527302 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.526768    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:04.527302 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.526790    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:04.527302 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.526803    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:04.527302 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.526865    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:06.526846001 +0000 UTC m=+6.096908813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:04.845635 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.845569    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:52:02 +0000 UTC" deadline="2028-01-07 09:43:27.188122417 +0000 UTC"
Jul 28 17:57:04.845635 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.845634    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12663h46m22.342492657s"
Jul 28 17:57:04.959345 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.958816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:04.959345 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:04.958951    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:04.997683 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:04.997646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal" event={"ID":"505a36e67b1ce4359d0b61ae9f5542fa","Type":"ContainerStarted","Data":"e531111df4d7d93105d14eea8b12176e0e55c9321253f1a8170886d5326d3313"}
Jul 28 17:57:05.008366 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:05.008262    2581 generic.go:356] "Generic (PLEG): container finished" podID="7312aca751091fa0f1482dbad6f5763c" containerID="e167bb209283fb784223edd5ee22927f4cd4294569241189aebfcd92be78c7c1" exitCode=0
Jul 28 17:57:05.008524 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:05.008377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal" event={"ID":"7312aca751091fa0f1482dbad6f5763c","Type":"ContainerDied","Data":"e167bb209283fb784223edd5ee22927f4cd4294569241189aebfcd92be78c7c1"}
Jul 28 17:57:05.014368 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:05.013955    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-244.ec2.internal" podStartSLOduration=3.013936663 podStartE2EDuration="3.013936663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:05.013504186 +0000 UTC m=+4.583567020" watchObservedRunningTime="2026-07-28 17:57:05.013936663 +0000 UTC m=+4.583999497"
Jul 28 17:57:05.958360 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:05.958325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:05.958811 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:05.958463    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:05.959329 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:05.958874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:05.959329 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:05.958993    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:06.023385 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:06.023341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal" event={"ID":"7312aca751091fa0f1482dbad6f5763c","Type":"ContainerStarted","Data":"a79da2b63e42e18498dd5b4bf85067090addc03fedee4f342effc0c84d7c4313"}
Jul 28 17:57:06.444374 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:06.444322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:06.444546 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:06.444389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:06.444546 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.444493    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:06.444546 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.444543    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:06.444708 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.444569    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:10.444547421 +0000 UTC m=+10.014610243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:06.444708 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.444594    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:10.444578641 +0000 UTC m=+10.014641467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:06.546325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:06.545612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:06.546325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.545843    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:06.546325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.545866    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:06.546325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.545891    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:06.546325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.545949    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:10.545930932 +0000 UTC m=+10.115993757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:06.958838 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:06.958373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:06.958838 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:06.958497    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:07.959356 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:07.958819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:07.959356 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:07.958954    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:07.959356 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:07.959314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:07.959921 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:07.959429    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:08.958640 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:08.958604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:08.958846 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:08.958755    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:09.958399 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:09.958318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:09.958916 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:09.958453    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:09.958916 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:09.958498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:09.958916 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:09.958618    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:10.479431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:10.479484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.479603    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.479669    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:18.47964975 +0000 UTC m=+18.049712561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.479666    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:10.479993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.479730    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:18.479712034 +0000 UTC m=+18.049774847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:10.580050 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:10.579989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:10.580211 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.580194    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:10.580315 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.580222    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:10.580315 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.580236    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:10.580411 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.580316    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:18.5802956 +0000 UTC m=+18.150358431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:10.959407 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:10.959368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:10.959861 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:10.959487    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:11.958583 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:11.958549    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:11.958742 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:11.958683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:11.958857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:11.958791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:11.958982 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:11.958952    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:12.960637 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:12.960605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:12.961119 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:12.960740    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:13.958715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:13.958680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:13.958912 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:13.958686    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:13.958912 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:13.958838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:13.958912 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:13.958893    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:14.958355 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:14.958323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:14.958831 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:14.958460    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:15.958409 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:15.958297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:15.958409 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:15.958345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:15.958857 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:15.958418    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:15.958857 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:15.958549    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:16.958394 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:16.958357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:16.958581 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:16.958475    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:17.958918 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:17.958874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:17.959397 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:17.958930    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:17.959397 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:17.959023    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:17.959397 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:17.959136    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:18.547748 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:18.547712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:18.547748 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:18.547755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:18.547961 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.547878    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:18.547961 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.547935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:57:34.547920352 +0000 UTC m=+34.117983160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:18.548092 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.547878    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:18.548092 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.548023    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:34.548008576 +0000 UTC m=+34.118071384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:18.649094 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:18.649008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:18.649266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.649200    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:18.649266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.649224    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:18.649266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.649235    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:18.649452 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.649312    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:34.649290292 +0000 UTC m=+34.219353102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:18.958618 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:18.958541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:18.958765 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:18.958659    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:19.958433 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:19.958384    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:19.958433 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:19.958413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:19.958941 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:19.958541    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:19.958941 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:19.958676    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:20.959420 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:20.959385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:20.959852 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:20.959481    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:21.958482 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:21.958449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:21.958643 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:21.958456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:21.958643 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:21.958554    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:21.958722 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:21.958673    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:22.958684 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:22.958372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:22.959496 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:22.958787    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:23.060173 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.060099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sl47n" event={"ID":"2e67f7db-91a9-4e7f-bfcc-21b79228466d","Type":"ContainerStarted","Data":"bb018c4d4be365d320f6902ab77865500788578dc8394ac032c873a0bab6dec9"}
Jul 28 17:57:23.061828 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.061802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kc972" event={"ID":"7ac26091-4d06-4f7a-bcf7-acde7784cc74","Type":"ContainerStarted","Data":"21316a3b1d8199888e01aaf937be00687ae8dfbbb4584d9ee3cbac67b101ea5f"}
Jul 28 17:57:23.064040 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.063773    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb" event={"ID":"9e960f89-a33d-4495-b827-70de2eddc0ab","Type":"ContainerStarted","Data":"077dc3fb418bb7d98d2073e87a9433da89e4f9c0456b533dc449135615b587f5"}
Jul 28 17:57:23.065370 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.065341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2shpt" event={"ID":"2ed0a3f3-869e-4539-87b3-9a7cbda2f78e","Type":"ContainerStarted","Data":"e1124faba320324a51d6e300cbfe242e243afde81bf2b0c9eb7529e2d5d07e28"}
Jul 28 17:57:23.067130 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.067107    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2c7f8" event={"ID":"bd0b3a5a-e281-410c-a2f8-a1c8e1fc6e94","Type":"ContainerStarted","Data":"39cb803aa88c84ca7a5fcf0aabe1a5f6c11d25442cd30f0862c362e2be678de5"}
Jul 28 17:57:23.070908 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.070882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"a4a636f38cbd271fded4fda860ca482bde439f515acc803063331606b805c26c"}
Jul 28 17:57:23.071032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.070959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"23435eecbc00cb2b8290ac662e65f55aedc63cbf3b2eabe4010e302a091a301e"}
Jul 28 17:57:23.071032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.071008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"5f27b2185c437554d4d24e66d73a716cfc445ee8af30ea7c9c32b853c9f3a608"}
Jul 28 17:57:23.071159 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.071061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"92d16edbe44b5319a0d649b74ab7fd01e6dcdfedfa12114e72edfa4270db82b0"}
Jul 28 17:57:23.071159 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.071112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"cf46974d9d5abab6c4ff88cd5021d6a3e1934a8469b930bc05628c837654e6d9"}
Jul 28 17:57:23.071255 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.071163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"0a6a644315749aa459e5d25b1e8ff0c55cf838199ad21eeee06d866c95fdf626"}
Jul 28 17:57:23.072479 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.072457    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="76d21b8a8fda24b8b9579dbaa13c270b8ea4cea50b24656935e76af1f0f91d06" exitCode=0
Jul 28 17:57:23.072599 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.072524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"76d21b8a8fda24b8b9579dbaa13c270b8ea4cea50b24656935e76af1f0f91d06"}
Jul 28 17:57:23.074499 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.074475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m9m2v" event={"ID":"15fec211-1790-4736-a469-069077d4c7fa","Type":"ContainerStarted","Data":"5feafc5dd287f8b16d8fcc89fccdeda56f528e47f2a2c2749c423ef5fad61fd4"}
Jul 28 17:57:23.077244 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.077196    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sl47n" podStartSLOduration=3.656554845 podStartE2EDuration="22.077178943s" totalImagesPullingTime="18.420624098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:23.076518807 +0000 UTC m=+22.646581639" watchObservedRunningTime="2026-07-28 17:57:23.077178943 +0000 UTC m=+22.647241776"
Jul 28 17:57:23.077369 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.077307    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-244.ec2.internal" podStartSLOduration=21.077300982 podStartE2EDuration="21.077300982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:06.039324318 +0000 UTC m=+5.609387154" watchObservedRunningTime="2026-07-28 17:57:23.077300982 +0000 UTC m=+22.647363812"
Jul 28 17:57:23.119632 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.119581    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2c7f8" podStartSLOduration=3.661474809 podStartE2EDuration="22.119564737s" totalImagesPullingTime="18.458089928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:23.119404215 +0000 UTC m=+22.689467045" watchObservedRunningTime="2026-07-28 17:57:23.119564737 +0000 UTC m=+22.689627567"
Jul 28 17:57:23.120018 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.119997    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kc972" podStartSLOduration=4.693449776 podStartE2EDuration="23.119988771s" totalImagesPullingTime="18.426538995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:23.097550573 +0000 UTC m=+22.667613406" watchObservedRunningTime="2026-07-28 17:57:23.119988771 +0000 UTC m=+22.690051602"
Jul 28 17:57:23.167983 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.167936    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2shpt" podStartSLOduration=3.668937309 podStartE2EDuration="22.167924607s" totalImagesPullingTime="18.498987298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:23.167357899 +0000 UTC m=+22.737420729" watchObservedRunningTime="2026-07-28 17:57:23.167924607 +0000 UTC m=+22.737987439"
Jul 28 17:57:23.182573 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.182516    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m9m2v" podStartSLOduration=7.672881842 podStartE2EDuration="22.182499609s" totalImagesPullingTime="14.509617767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:23.18234069 +0000 UTC m=+22.752403521" watchObservedRunningTime="2026-07-28 17:57:23.182499609 +0000 UTC m=+22.752562439"
Jul 28 17:57:23.715929 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.715903    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 17:57:23.864114 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.863985    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:57:23.715923286Z","UUID":"bacb775e-9d31-48e6-b092-f4da3d7b02bf","Handler":null,"Name":"","Endpoint":""}
Jul 28 17:57:23.865621 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.865599    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 17:57:23.865734 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.865638    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 17:57:23.958421 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.958332    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:23.958421 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:23.958372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:23.958738 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:23.958457    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:23.958738 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:23.958573    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:24.079358 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:24.079307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb" event={"ID":"9e960f89-a33d-4495-b827-70de2eddc0ab","Type":"ContainerStarted","Data":"05449e86111d9f2375c5f6424e5dfa4526539a1f4da51b2d8ba41ea78e7d22b5"}
Jul 28 17:57:24.081053 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:24.080983    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8m4k" event={"ID":"1f621ff1-7734-4d8b-b424-583b35762678","Type":"ContainerStarted","Data":"9704ace5940c2c1d0a9e4c45cadaaaae38d47c96f1187440536b16bdea374667"}
Jul 28 17:57:24.098307 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:24.098241    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s8m4k" podStartSLOduration=4.676442267 podStartE2EDuration="23.098221933s" totalImagesPullingTime="18.421779666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:24.098224149 +0000 UTC m=+23.668286981" watchObservedRunningTime="2026-07-28 17:57:24.098221933 +0000 UTC m=+23.668284746"
Jul 28 17:57:24.958312 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:24.958286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:24.958475 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:24.958422    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:25.085828 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:25.085787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb" event={"ID":"9e960f89-a33d-4495-b827-70de2eddc0ab","Type":"ContainerStarted","Data":"45cc93ae043949852afc77e0482c5d8f27e5bc918ec3b6d454627611ed8e81e6"}
Jul 28 17:57:25.106006 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:25.105963    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9lpmb" podStartSLOduration=4.431461928 podStartE2EDuration="25.105946791s" totalImagesPullingTime="20.674484863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:25.105361508 +0000 UTC m=+24.675424339" watchObservedRunningTime="2026-07-28 17:57:25.105946791 +0000 UTC m=+24.676009622"
Jul 28 17:57:25.958835 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:25.958797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:25.958835 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:25.958819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:25.959046 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:25.958939    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:25.959108 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:25.959045    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:26.092335 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:26.092284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"6bcdb747e6becdd2ac9b2822bee170ef0c0c59dd401ff05b4a44562127058f06"}
Jul 28 17:57:26.218638 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:26.218545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:26.219457 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:26.219435    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:26.958285 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:26.958243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:26.958477 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:26.958389    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:27.614300 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:27.614090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:27.614986 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:27.614689    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m9m2v"
Jul 28 17:57:27.957974 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:27.957949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:27.957974 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:27.957962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:27.958201 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:27.958061    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:27.958201 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:27.958169    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:28.100444 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.100407    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" event={"ID":"b3373d24-6d72-4a4b-8be6-bb3604dd1c5a","Type":"ContainerStarted","Data":"0d4f56b9092e8a5c8a402238430eb69dc50ff261a240deedddfc8a229ef7dbc2"}
Jul 28 17:57:28.100646 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.100622    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:28.100838 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.100766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:28.102471 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.102445    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="1fc5604ae3894c613de44727e8f57cda162ccfb3d4be6d3847286d0575bb47a3" exitCode=0
Jul 28 17:57:28.102602 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.102514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"1fc5604ae3894c613de44727e8f57cda162ccfb3d4be6d3847286d0575bb47a3"}
Jul 28 17:57:28.116221 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.116198    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:28.116363 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.116350    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:28.187814 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.187755    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg" podStartSLOduration=8.695191368 podStartE2EDuration="27.187737811s" totalImagesPullingTime="18.492546443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:28.137227782 +0000 UTC m=+27.707290614" watchObservedRunningTime="2026-07-28 17:57:28.187737811 +0000 UTC m=+27.757800620"
Jul 28 17:57:28.431061 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.431024    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:57:28.958367 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:28.958335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:28.958808 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:28.958449    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:29.763398 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.763320    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lwbk"]
Jul 28 17:57:29.763614 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.763436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:29.763614 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:29.763539    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:29.766732 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.766694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqhrw"]
Jul 28 17:57:29.766897 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.766878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:29.767042 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:29.767015    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:29.767466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.767444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsqvb"]
Jul 28 17:57:29.767552 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:29.767538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:29.767651 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:29.767632    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:30.108185 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:30.108150    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="12be70da8719bb636d5c6dfea471ccfe8dfc68c1230874cb9187f8c826b480e8" exitCode=0
Jul 28 17:57:30.108651 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:30.108232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"12be70da8719bb636d5c6dfea471ccfe8dfc68c1230874cb9187f8c826b480e8"}
Jul 28 17:57:31.958214 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:31.958022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:31.958653 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:31.958022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:31.958653 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:31.958303    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:31.958653 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:31.958022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:31.958653 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:31.958387    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:31.958653 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:31.958466    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:32.114027 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:32.113990    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="ce0e122cb7fbc16fa5e1cc7a0211a11c1d897787423e4a8d87089eadaf99e97c" exitCode=0
Jul 28 17:57:32.114217 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:32.114037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"ce0e122cb7fbc16fa5e1cc7a0211a11c1d897787423e4a8d87089eadaf99e97c"}
Jul 28 17:57:33.958634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:33.958602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:33.958634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:33.958633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:33.959105 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:33.958731    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqhrw" podUID="5dc72e56-2e79-423c-aca1-db45ca996cda"
Jul 28 17:57:33.959105 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:33.958768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:33.959105 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:33.958902    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsqvb" podUID="566d6462-3b3d-410b-b27a-34a0d1a74d15"
Jul 28 17:57:33.959105 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:33.958976    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8lwbk" podUID="bcfd65e2-69dc-4593-aa86-aa6a8c84c366"
Jul 28 17:57:34.565067 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:34.565022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:34.565266 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:34.565126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:34.565266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.565191    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:34.565266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.565212    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:34.565266 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.565281    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:58:06.565252978 +0000 UTC m=+66.135315790 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 17:57:34.565487 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.565299    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret podName:bcfd65e2-69dc-4593-aa86-aa6a8c84c366 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:06.565290235 +0000 UTC m=+66.135353046 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret") pod "global-pull-secret-syncer-8lwbk" (UID: "bcfd65e2-69dc-4593-aa86-aa6a8c84c366") : object "kube-system"/"original-pull-secret" not registered
Jul 28 17:57:34.665815 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:34.665777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:34.665993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.665909    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 17:57:34.665993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.665932    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 17:57:34.665993 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.665944    2581 projected.go:196] Error preparing data for projected volume kube-api-access-m985w for pod openshift-network-diagnostics/network-check-target-tsqvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:34.666099 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:34.665996    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w podName:566d6462-3b3d-410b-b27a-34a0d1a74d15 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:06.665983603 +0000 UTC m=+66.236046412 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m985w" (UniqueName: "kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w") pod "network-check-target-tsqvb" (UID: "566d6462-3b3d-410b-b27a-34a0d1a74d15") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 17:57:35.279255 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.279222    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-244.ec2.internal" event="NodeReady"
Jul 28 17:57:35.279852 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.279407    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 17:57:35.320594 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.320557    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7k8k8"]
Jul 28 17:57:35.323705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.323375    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.324179 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.324154    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hl8"]
Jul 28 17:57:35.326372 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.326141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.326372 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.326252    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5mqwj\""
Jul 28 17:57:35.326372 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.326264    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 17:57:35.326555 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.326451    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 17:57:35.329149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.328681    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 17:57:35.329149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.328743    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 17:57:35.329149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.328681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vjgn2\""
Jul 28 17:57:35.329149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.328970    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 17:57:35.334991 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.334350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7k8k8"]
Jul 28 17:57:35.338490 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.338365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hl8"]
Jul 28 17:57:35.473349 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-cert\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.473349 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvdg5\" (UniqueName: \"kubernetes.io/projected/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-kube-api-access-jvdg5\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.473587 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.473587 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l89l\" (UniqueName: \"kubernetes.io/projected/35236c73-366b-4c06-8832-56a0790f2282-kube-api-access-4l89l\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.473587 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35236c73-366b-4c06-8832-56a0790f2282-config-volume\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.473704 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.473617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35236c73-366b-4c06-8832-56a0790f2282-tmp-dir\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.574753 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35236c73-366b-4c06-8832-56a0790f2282-config-volume\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.574920 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35236c73-366b-4c06-8832-56a0790f2282-tmp-dir\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.574920 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-cert\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.574920 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvdg5\" (UniqueName: \"kubernetes.io/projected/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-kube-api-access-jvdg5\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.574920 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.574920 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.574911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l89l\" (UniqueName: \"kubernetes.io/projected/35236c73-366b-4c06-8832-56a0790f2282-kube-api-access-4l89l\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.575162 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:35.575084    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:35.575234 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:35.575185    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:36.075159487 +0000 UTC m=+35.645222319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:35.575234 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.575220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35236c73-366b-4c06-8832-56a0790f2282-tmp-dir\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.575417 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.575392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/35236c73-366b-4c06-8832-56a0790f2282-config-volume\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.579485 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.579463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-cert\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.586315 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.586261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l89l\" (UniqueName: \"kubernetes.io/projected/35236c73-366b-4c06-8832-56a0790f2282-kube-api-access-4l89l\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:35.586510 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.586489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvdg5\" (UniqueName: \"kubernetes.io/projected/8dea7d38-b4c7-49a8-80a3-c86f9e10e06b-kube-api-access-jvdg5\") pod \"ingress-canary-g2hl8\" (UID: \"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b\") " pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.645485 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.645446    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2hl8"
Jul 28 17:57:35.958907 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.958877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:57:35.959023 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.958928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:57:35.959023 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.958876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:57:35.961544 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.961505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pbl4v\""
Jul 28 17:57:35.962537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.962236    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:57:35.962537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.962252    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 17:57:35.962537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.962251    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgwbs\""
Jul 28 17:57:35.962537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.962333    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:57:35.962537 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:35.962251    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 17:57:36.079366 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:36.079118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:36.079470 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:36.079244    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:36.079536 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:36.079521    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:37.079499152 +0000 UTC m=+36.649561977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:36.089175 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:36.089145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hl8"]
Jul 28 17:57:36.093309 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:36.093244    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dea7d38_b4c7_49a8_80a3_c86f9e10e06b.slice/crio-876e3fb80c0e72b08b63ff88a0764086168b828f2f6b52d4fa5c697ce1df7491 WatchSource:0}: Error finding container 876e3fb80c0e72b08b63ff88a0764086168b828f2f6b52d4fa5c697ce1df7491: Status 404 returned error can't find the container with id 876e3fb80c0e72b08b63ff88a0764086168b828f2f6b52d4fa5c697ce1df7491
Jul 28 17:57:36.127363 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:36.127329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerStarted","Data":"aba99733242e428b1eb64ca4e1d85518475271540f28d728d36f5478d7c17eab"}
Jul 28 17:57:36.128434 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:36.128406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2hl8" event={"ID":"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b","Type":"ContainerStarted","Data":"876e3fb80c0e72b08b63ff88a0764086168b828f2f6b52d4fa5c697ce1df7491"}
Jul 28 17:57:37.088986 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:37.088951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:37.089618 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:37.089113    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:37.089618 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:37.089189    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:39.089172499 +0000 UTC m=+38.659235327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:37.134460 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:37.134416    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="aba99733242e428b1eb64ca4e1d85518475271540f28d728d36f5478d7c17eab" exitCode=0
Jul 28 17:57:37.134630 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:37.134469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"aba99733242e428b1eb64ca4e1d85518475271540f28d728d36f5478d7c17eab"}
Jul 28 17:57:38.140317 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:38.140253    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a4b1fd7-dad0-4eb2-88dc-bf34529844e7" containerID="307de00374ab7a8c277c4d345e8c108d6f8ca68f981bbc2f5c38af7beae76da8" exitCode=0
Jul 28 17:57:38.140796 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:38.140319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerDied","Data":"307de00374ab7a8c277c4d345e8c108d6f8ca68f981bbc2f5c38af7beae76da8"}
Jul 28 17:57:39.102327 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:39.102125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:39.102439 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:39.102293    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:39.102480 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:39.102443    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:43.102427203 +0000 UTC m=+42.672490015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:39.146501 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:39.146469    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" event={"ID":"6a4b1fd7-dad0-4eb2-88dc-bf34529844e7","Type":"ContainerStarted","Data":"da71901af1b4b2552f2652a47070420a822c42bbb497703ba892f06881fb2dfe"}
Jul 28 17:57:39.147969 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:39.147944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2hl8" event={"ID":"8dea7d38-b4c7-49a8-80a3-c86f9e10e06b","Type":"ContainerStarted","Data":"a7ea4e4b0a6836110081b6b613fd109089737eea47c7d4c495c4178f165d76f8"}
Jul 28 17:57:39.170085 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:39.170039    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jrf4c" podStartSLOduration=9.165706871 podStartE2EDuration="38.170024404s" totalImagesPullingTime="29.004317533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:39.169261235 +0000 UTC m=+38.739324079" watchObservedRunningTime="2026-07-28 17:57:39.170024404 +0000 UTC m=+38.740087225"
Jul 28 17:57:39.187395 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:39.187344    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g2hl8" podStartSLOduration=1.448318241 podStartE2EDuration="4.187328058s" totalImagesPullingTime="2.739009817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:39.186638234 +0000 UTC m=+38.756701064" watchObservedRunningTime="2026-07-28 17:57:39.187328058 +0000 UTC m=+38.757390888"
Jul 28 17:57:43.129481 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:43.129439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:43.129919 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:43.129603    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:43.129919 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:43.129668    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:51.129651111 +0000 UTC m=+50.699713940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:48.558875 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.558840    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"]
Jul 28 17:57:48.596476 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.596441    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:57:48.596652 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.596583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"
Jul 28 17:57:48.598813 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.598787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-n97nm\""
Jul 28 17:57:48.612024 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.611994    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"]
Jul 28 17:57:48.612024 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.612032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:57:48.612206 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.612104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.614346 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.614326    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 17:57:48.614529 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.614451    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 17:57:48.614529 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.614507    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vrsc6\""
Jul 28 17:57:48.614529 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.614513    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 17:57:48.620845 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.620822    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 17:57:48.667398 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.667363    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zhnzc"]
Jul 28 17:57:48.688763 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.688728    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"]
Jul 28 17:57:48.688938 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.688860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.691409 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.691389    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 17:57:48.691638 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.691612    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-72jwd\""
Jul 28 17:57:48.691763 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.691710    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 17:57:48.691848 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.691784    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.691945 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.691930    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.696173 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.696075    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 17:57:48.703135 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.703107    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zhnzc"]
Jul 28 17:57:48.703135 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.703120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.703307 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.703151    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"]
Jul 28 17:57:48.705257 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.705239    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-klztn\""
Jul 28 17:57:48.705365 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.705238    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.705624 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.705587    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 17:57:48.705714 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.705693    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 17:57:48.706591 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.706575    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.763125 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.763089    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"]
Jul 28 17:57:48.768130 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768213 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768213 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768213 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768391 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768391 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768490 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xthdg\" (UniqueName: \"kubernetes.io/projected/633f8683-61e2-4741-9fc7-464da52425e7-kube-api-access-xthdg\") pod \"network-check-source-7d88cb55d8-68kn5\" (UID: \"633f8683-61e2-4741-9fc7-464da52425e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"
Jul 28 17:57:48.768490 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.768490 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.768482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqxl6\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.783336 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.783308    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"]
Jul 28 17:57:48.783467 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.783361    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.785560 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.785541    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 17:57:48.785688 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.785655    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d82vv\""
Jul 28 17:57:48.785963 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.785943    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 17:57:48.786640 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.786619    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 17:57:48.786714 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.786678    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 17:57:48.869217 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.869425 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xthdg\" (UniqueName: \"kubernetes.io/projected/633f8683-61e2-4741-9fc7-464da52425e7-kube-api-access-xthdg\") pod \"network-check-source-7d88cb55d8-68kn5\" (UID: \"633f8683-61e2-4741-9fc7-464da52425e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"
Jul 28 17:57:48.869425 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/631859d9-19f5-4e56-9af5-cf1f161e40fd-serving-cert\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.869425 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.869425 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0394e7da-58ea-4122-87bb-ee65437b2e6c-config\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.869425 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjf8\" (UniqueName: \"kubernetes.io/projected/631859d9-19f5-4e56-9af5-cf1f161e40fd-kube-api-access-shjf8\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487wz\" (UniqueName: \"kubernetes.io/projected/0394e7da-58ea-4122-87bb-ee65437b2e6c-kube-api-access-487wz\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0394e7da-58ea-4122-87bb-ee65437b2e6c-serving-cert\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-service-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-snapshots\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.869709 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqxl6\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:48.869787    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:48.869804    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b578869dd-lg22w: secret "image-registry-tls" not found
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:48.869855    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls podName:207cc8b2-bbc8-4b05-b73d-d41921803c96 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.369835294 +0000 UTC m=+48.939898102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls") pod "image-registry-6b578869dd-lg22w" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96") : secret "image-registry-tls" not found
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff634b17-0e6a-456b-b337-2db00422219d-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870032 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.869993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgtx2\" (UniqueName: \"kubernetes.io/projected/ff634b17-0e6a-456b-b337-2db00422219d-kube-api-access-zgtx2\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.870494 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.870047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870494 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.870101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-tmp\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.870583 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.870502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.870828 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.870808    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.873603 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.873583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.873715 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.873618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.882604 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.882578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xthdg\" (UniqueName: \"kubernetes.io/projected/633f8683-61e2-4741-9fc7-464da52425e7-kube-api-access-xthdg\") pod \"network-check-source-7d88cb55d8-68kn5\" (UID: \"633f8683-61e2-4741-9fc7-464da52425e7\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"
Jul 28 17:57:48.882711 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.882636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.884095 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.884072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqxl6\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:48.905967 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.905942    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"
Jul 28 17:57:48.970626 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0394e7da-58ea-4122-87bb-ee65437b2e6c-config\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.970626 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjf8\" (UniqueName: \"kubernetes.io/projected/631859d9-19f5-4e56-9af5-cf1f161e40fd-kube-api-access-shjf8\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487wz\" (UniqueName: \"kubernetes.io/projected/0394e7da-58ea-4122-87bb-ee65437b2e6c-kube-api-access-487wz\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0394e7da-58ea-4122-87bb-ee65437b2e6c-serving-cert\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-service-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-snapshots\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff634b17-0e6a-456b-b337-2db00422219d-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.970935 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgtx2\" (UniqueName: \"kubernetes.io/projected/ff634b17-0e6a-456b-b337-2db00422219d-kube-api-access-zgtx2\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.971685 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.970975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-tmp\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.971685 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.971005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.971685 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.971038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/631859d9-19f5-4e56-9af5-cf1f161e40fd-serving-cert\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.971685 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:48.971593    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:48.971685 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:48.971671    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:49.471649992 +0000 UTC m=+49.041712802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:48.971948 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.971826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-tmp\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.972598 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.972158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0394e7da-58ea-4122-87bb-ee65437b2e6c-config\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.972598 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.972251    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-service-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.972598 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.972530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/631859d9-19f5-4e56-9af5-cf1f161e40fd-snapshots\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.972598 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.972529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631859d9-19f5-4e56-9af5-cf1f161e40fd-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.972966 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.972906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff634b17-0e6a-456b-b337-2db00422219d-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.974192 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.974168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/631859d9-19f5-4e56-9af5-cf1f161e40fd-serving-cert\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.974311 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.974176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0394e7da-58ea-4122-87bb-ee65437b2e6c-serving-cert\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.983238 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.983207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjf8\" (UniqueName: \"kubernetes.io/projected/631859d9-19f5-4e56-9af5-cf1f161e40fd-kube-api-access-shjf8\") pod \"insights-operator-5885fdb759-zhnzc\" (UID: \"631859d9-19f5-4e56-9af5-cf1f161e40fd\") " pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:48.983470 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.983446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-487wz\" (UniqueName: \"kubernetes.io/projected/0394e7da-58ea-4122-87bb-ee65437b2e6c-kube-api-access-487wz\") pod \"service-ca-operator-5d6c75f84f-z9sp9\" (UID: \"0394e7da-58ea-4122-87bb-ee65437b2e6c\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:48.983567 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.983546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgtx2\" (UniqueName: \"kubernetes.io/projected/ff634b17-0e6a-456b-b337-2db00422219d-kube-api-access-zgtx2\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:48.998489 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:48.998455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zhnzc"
Jul 28 17:57:49.011599 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.011569    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"
Jul 28 17:57:49.059818 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.059708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5"]
Jul 28 17:57:49.065545 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:49.065512    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633f8683_61e2_4741_9fc7_464da52425e7.slice/crio-c672d218a3c98354ee028f01b5ff1298c9088f7893a1931f4aa500366d4b1a37 WatchSource:0}: Error finding container c672d218a3c98354ee028f01b5ff1298c9088f7893a1931f4aa500366d4b1a37: Status 404 returned error can't find the container with id c672d218a3c98354ee028f01b5ff1298c9088f7893a1931f4aa500366d4b1a37
Jul 28 17:57:49.171952 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.171917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5" event={"ID":"633f8683-61e2-4741-9fc7-464da52425e7","Type":"ContainerStarted","Data":"c672d218a3c98354ee028f01b5ff1298c9088f7893a1931f4aa500366d4b1a37"}
Jul 28 17:57:49.182595 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.182550    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zhnzc"]
Jul 28 17:57:49.186223 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:49.186199    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631859d9_19f5_4e56_9af5_cf1f161e40fd.slice/crio-aa06f618b818598ecdeeb327c29a08c33e6b22eafb4c9e87888d605fa005a1a1 WatchSource:0}: Error finding container aa06f618b818598ecdeeb327c29a08c33e6b22eafb4c9e87888d605fa005a1a1: Status 404 returned error can't find the container with id aa06f618b818598ecdeeb327c29a08c33e6b22eafb4c9e87888d605fa005a1a1
Jul 28 17:57:49.200913 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.200885    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9"]
Jul 28 17:57:49.203808 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:57:49.203783    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0394e7da_58ea_4122_87bb_ee65437b2e6c.slice/crio-b126f9ae8d27c01c0adc8d1bc86f4bd29e867fd93992c8424c44a5b1b6293519 WatchSource:0}: Error finding container b126f9ae8d27c01c0adc8d1bc86f4bd29e867fd93992c8424c44a5b1b6293519: Status 404 returned error can't find the container with id b126f9ae8d27c01c0adc8d1bc86f4bd29e867fd93992c8424c44a5b1b6293519
Jul 28 17:57:49.375851 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.375814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:49.376015 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:49.375969    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:57:49.376015 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:49.375989    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b578869dd-lg22w: secret "image-registry-tls" not found
Jul 28 17:57:49.376080 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:49.376044    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls podName:207cc8b2-bbc8-4b05-b73d-d41921803c96 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.376028909 +0000 UTC m=+49.946091719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls") pod "image-registry-6b578869dd-lg22w" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96") : secret "image-registry-tls" not found
Jul 28 17:57:49.476595 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:49.476505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:49.476740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:49.476635    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:49.476740 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:49.476690    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:50.476674685 +0000 UTC m=+50.046737494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:50.176634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:50.176556    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9" event={"ID":"0394e7da-58ea-4122-87bb-ee65437b2e6c","Type":"ContainerStarted","Data":"b126f9ae8d27c01c0adc8d1bc86f4bd29e867fd93992c8424c44a5b1b6293519"}
Jul 28 17:57:50.178001 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:50.177953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zhnzc" event={"ID":"631859d9-19f5-4e56-9af5-cf1f161e40fd","Type":"ContainerStarted","Data":"aa06f618b818598ecdeeb327c29a08c33e6b22eafb4c9e87888d605fa005a1a1"}
Jul 28 17:57:50.385600 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:50.385559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:50.385845 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:50.385819    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:57:50.385983 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:50.385854    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b578869dd-lg22w: secret "image-registry-tls" not found
Jul 28 17:57:50.385983 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:50.385921    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls podName:207cc8b2-bbc8-4b05-b73d-d41921803c96 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.385900864 +0000 UTC m=+51.955963702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls") pod "image-registry-6b578869dd-lg22w" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96") : secret "image-registry-tls" not found
Jul 28 17:57:50.487174 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:50.486479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:50.487174 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:50.486648    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:50.487174 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:50.486717    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:52.486696711 +0000 UTC m=+52.056759524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:51.193892 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:51.193853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:57:51.194339 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:51.194010    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 17:57:51.194339 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:51.194074    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls podName:35236c73-366b-4c06-8832-56a0790f2282 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:07.194058386 +0000 UTC m=+66.764121197 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls") pod "dns-default-7k8k8" (UID: "35236c73-366b-4c06-8832-56a0790f2282") : secret "dns-default-metrics-tls" not found
Jul 28 17:57:52.405748 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:52.405670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:52.406234 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:52.405818    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:57:52.406234 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:52.405844    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b578869dd-lg22w: secret "image-registry-tls" not found
Jul 28 17:57:52.406234 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:52.405909    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls podName:207cc8b2-bbc8-4b05-b73d-d41921803c96 nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.405892933 +0000 UTC m=+55.975955742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls") pod "image-registry-6b578869dd-lg22w" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96") : secret "image-registry-tls" not found
Jul 28 17:57:52.506187 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:52.506146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:52.506388 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:52.506312    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:52.506388 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:52.506377    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.506359075 +0000 UTC m=+56.076421902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:54.090785 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.090750    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"]
Jul 28 17:57:54.104439 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.104412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"
Jul 28 17:57:54.106640 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.106572    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"]
Jul 28 17:57:54.107860 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.107834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6k2vn\""
Jul 28 17:57:54.109230 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.108937    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 17:57:54.109230 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.108940    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 17:57:54.120816 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.120792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvmkk\" (UniqueName: \"kubernetes.io/projected/4388985f-f081-4bae-b679-3b9b1f1fefd1-kube-api-access-rvmkk\") pod \"migrator-6865c9967-x9mr4\" (UID: \"4388985f-f081-4bae-b679-3b9b1f1fefd1\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"
Jul 28 17:57:54.189698 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.189660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9" event={"ID":"0394e7da-58ea-4122-87bb-ee65437b2e6c","Type":"ContainerStarted","Data":"f4c7bda75dd8c9e69c6c78e8a88b354e4ce1d2ea8af462d312b4af510eec77e1"}
Jul 28 17:57:54.191293 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.191243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zhnzc" event={"ID":"631859d9-19f5-4e56-9af5-cf1f161e40fd","Type":"ContainerStarted","Data":"fdbb69b844bf4ad0a8bdea8465845cb8522091780a33eef5668c7bb587bc5aa6"}
Jul 28 17:57:54.192740 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.192706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5" event={"ID":"633f8683-61e2-4741-9fc7-464da52425e7","Type":"ContainerStarted","Data":"71fc7b3f77f5289b51e3e634f438fc839963e4586087c3a5070e6c15bc7e71fe"}
Jul 28 17:57:54.208522 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.208479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9" podStartSLOduration=2.044215439 podStartE2EDuration="6.20846602s" totalImagesPullingTime="4.164250581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:54.207895285 +0000 UTC m=+53.777958115" watchObservedRunningTime="2026-07-28 17:57:54.20846602 +0000 UTC m=+53.778528848"
Jul 28 17:57:54.221868 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.221840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvmkk\" (UniqueName: \"kubernetes.io/projected/4388985f-f081-4bae-b679-3b9b1f1fefd1-kube-api-access-rvmkk\") pod \"migrator-6865c9967-x9mr4\" (UID: \"4388985f-f081-4bae-b679-3b9b1f1fefd1\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"
Jul 28 17:57:54.225833 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.225788    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-68kn5" podStartSLOduration=1.912770696 podStartE2EDuration="6.225774716s" totalImagesPullingTime="4.31300402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:54.225324621 +0000 UTC m=+53.795387453" watchObservedRunningTime="2026-07-28 17:57:54.225774716 +0000 UTC m=+53.795837580"
Jul 28 17:57:54.237470 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.237442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvmkk\" (UniqueName: \"kubernetes.io/projected/4388985f-f081-4bae-b679-3b9b1f1fefd1-kube-api-access-rvmkk\") pod \"migrator-6865c9967-x9mr4\" (UID: \"4388985f-f081-4bae-b679-3b9b1f1fefd1\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"
Jul 28 17:57:54.247725 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.247671    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-zhnzc" podStartSLOduration=2.065384365 podStartE2EDuration="6.247648541s" totalImagesPullingTime="4.182264176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:54.246584618 +0000 UTC m=+53.816647450" watchObservedRunningTime="2026-07-28 17:57:54.247648541 +0000 UTC m=+53.817711372"
Jul 28 17:57:54.416150 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.416043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"
Jul 28 17:57:54.574776 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:54.574741    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4"]
Jul 28 17:57:55.184281 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.184238    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6ndmx"]
Jul 28 17:57:55.186565 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.186536    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.188958 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.188938    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 17:57:55.189076 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.189016    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 17:57:55.189141 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.189072    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7kkv\""
Jul 28 17:57:55.198427 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.198324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ndmx"]
Jul 28 17:57:55.198427 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.198388    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4" event={"ID":"4388985f-f081-4bae-b679-3b9b1f1fefd1","Type":"ContainerStarted","Data":"bb40c5bf035e2c51592395221f7d666d68f3a5130c8acf1a450a83d340737726"}
Jul 28 17:57:55.230146 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.230114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f98f8ecb-0659-442b-9b35-eefff9f4007d-crio-socket\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.230341 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.230161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.230341 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.230297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f98f8ecb-0659-442b-9b35-eefff9f4007d-data-volume\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.230467 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.230343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgnrj\" (UniqueName: \"kubernetes.io/projected/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-api-access-xgnrj\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.230467 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.230385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.331807 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f98f8ecb-0659-442b-9b35-eefff9f4007d-crio-socket\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f98f8ecb-0659-442b-9b35-eefff9f4007d-data-volume\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f98f8ecb-0659-442b-9b35-eefff9f4007d-crio-socket\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgnrj\" (UniqueName: \"kubernetes.io/projected/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-api-access-xgnrj\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.331976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.332043 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.331980    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.332415 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.332059    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:55.832039443 +0000 UTC m=+55.402102284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.332415 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.332128    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.332415 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.332188    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:55.832172217 +0000 UTC m=+55.402235046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.332415 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.332264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f98f8ecb-0659-442b-9b35-eefff9f4007d-data-volume\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.343630 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.343600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgnrj\" (UniqueName: \"kubernetes.io/projected/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-api-access-xgnrj\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.835918 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.835883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.836070 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:55.835947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:55.836070 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.836001    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.836070 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.836059    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:55.836070 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.836065    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.836047177 +0000 UTC m=+56.406109985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:55.836205 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:55.836128    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:56.836087374 +0000 UTC m=+56.406150184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.203428 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.203388    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4" event={"ID":"4388985f-f081-4bae-b679-3b9b1f1fefd1","Type":"ContainerStarted","Data":"93f01f54840eece5f7e4fec228a57c00ea8028971d1965d7a5399866cf97064b"}
Jul 28 17:57:56.203902 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.203482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4" event={"ID":"4388985f-f081-4bae-b679-3b9b1f1fefd1","Type":"ContainerStarted","Data":"b9e6d1f234c37c04049dd7ce5ff10ff1e19aa1e548cc40b804a89bb832b56528"}
Jul 28 17:57:56.220830 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.220774    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-x9mr4" podStartSLOduration=1.095213629 podStartE2EDuration="2.220758974s" totalImagesPullingTime="1.125545345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:57:56.219953178 +0000 UTC m=+55.790016012" watchObservedRunningTime="2026-07-28 17:57:56.220758974 +0000 UTC m=+55.790821854"
Jul 28 17:57:56.441166 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.441127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:57:56.441400 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.441299    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 17:57:56.441400 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.441318    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b578869dd-lg22w: secret "image-registry-tls" not found
Jul 28 17:57:56.441400 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.441373    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls podName:207cc8b2-bbc8-4b05-b73d-d41921803c96 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.441358297 +0000 UTC m=+64.011421110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls") pod "image-registry-6b578869dd-lg22w" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96") : secret "image-registry-tls" not found
Jul 28 17:57:56.542221 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.542123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:57:56.542416 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.542262    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:56.542416 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.542360    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:04.542338342 +0000 UTC m=+64.112401176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:57:56.844113 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.844072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:56.844320 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:56.844197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:56.844320 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.844228    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.844320 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.844307    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:58.844291575 +0000 UTC m=+58.414354404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:56.844320 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.844318    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:56.844641 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:56.844370    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:57:58.844354728 +0000 UTC m=+58.414417538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:57.846669 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:57.846643    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kc972_7ac26091-4d06-4f7a-bcf7-acde7784cc74/dns-node-resolver/0.log"
Jul 28 17:57:58.861524 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:58.861483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:58.862046 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:58.861613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:57:58.862046 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:58.861641    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:58.862046 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:58.861704    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:02.861687625 +0000 UTC m=+62.431750434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:57:58.862046 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:58.861725    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:57:58.862046 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:57:58.861776    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:02.861762822 +0000 UTC m=+62.431825631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : configmap "kube-rbac-proxy" not found
Jul 28 17:57:59.246976 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:59.246897    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sl47n_2e67f7db-91a9-4e7f-bfcc-21b79228466d/node-ca/0.log"
Jul 28 17:57:59.647043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:57:59.647011    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g2hl8_8dea7d38-b4c7-49a8-80a3-c86f9e10e06b/serve-healthcheck-canary/0.log"
Jul 28 17:58:00.118519 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:00.118493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j9lbg"
Jul 28 17:58:00.251073 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:00.251036    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x9mr4_4388985f-f081-4bae-b679-3b9b1f1fefd1/migrator/0.log"
Jul 28 17:58:00.447137 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:00.447065    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x9mr4_4388985f-f081-4bae-b679-3b9b1f1fefd1/graceful-termination/0.log"
Jul 28 17:58:02.892335 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:02.892290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:02.892726 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:02.892389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:02.892726 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:02.892398    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 17:58:02.892726 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:02.892487    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:10.892467739 +0000 UTC m=+70.462530568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : configmap "kube-rbac-proxy" not found
Jul 28 17:58:02.892726 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:02.892511    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 17:58:02.892726 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:02.892558    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls podName:f98f8ecb-0659-442b-9b35-eefff9f4007d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:10.892547297 +0000 UTC m=+70.462610115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ndmx" (UID: "f98f8ecb-0659-442b-9b35-eefff9f4007d") : secret "insights-runtime-extractor-tls" not found
Jul 28 17:58:04.505263 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.505224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:04.507707 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.507682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"image-registry-6b578869dd-lg22w\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") " pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:04.524444 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.524409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vrsc6\""
Jul 28 17:58:04.532583 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.532555    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:04.606690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.606657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:58:04.606882 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:04.606817    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 17:58:04.606943 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:04.606885    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls podName:ff634b17-0e6a-456b-b337-2db00422219d nodeName:}" failed. No retries permitted until 2026-07-28 17:58:20.606869236 +0000 UTC m=+80.176932044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-t7586" (UID: "ff634b17-0e6a-456b-b337-2db00422219d") : secret "cluster-monitoring-operator-tls" not found
Jul 28 17:58:04.664541 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:04.664501    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:58:04.668329 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:04.668301    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207cc8b2_bbc8_4b05_b73d_d41921803c96.slice/crio-a48c65b661078fc9726a2f8f044a9ee956122b92dfe58122d315a24b4cb26352 WatchSource:0}: Error finding container a48c65b661078fc9726a2f8f044a9ee956122b92dfe58122d315a24b4cb26352: Status 404 returned error can't find the container with id a48c65b661078fc9726a2f8f044a9ee956122b92dfe58122d315a24b4cb26352
Jul 28 17:58:05.232287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:05.232236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" event={"ID":"207cc8b2-bbc8-4b05-b73d-d41921803c96","Type":"ContainerStarted","Data":"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"}
Jul 28 17:58:05.232287 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:05.232298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" event={"ID":"207cc8b2-bbc8-4b05-b73d-d41921803c96","Type":"ContainerStarted","Data":"a48c65b661078fc9726a2f8f044a9ee956122b92dfe58122d315a24b4cb26352"}
Jul 28 17:58:05.232549 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:05.232328    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:05.258924 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:05.258872    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" podStartSLOduration=17.258854735 podStartE2EDuration="17.258854735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:05.257539604 +0000 UTC m=+64.827602435" watchObservedRunningTime="2026-07-28 17:58:05.258854735 +0000 UTC m=+64.828917566"
Jul 28 17:58:06.624339 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.624287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:58:06.624742 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.624380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:58:06.629358 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.629327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 17:58:06.629492 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.629370    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 17:58:06.634562 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:06.634541    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 17:58:06.634679 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:06.634608    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs podName:5dc72e56-2e79-423c-aca1-db45ca996cda nodeName:}" failed. No retries permitted until 2026-07-28 17:59:10.634591521 +0000 UTC m=+130.204654330 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs") pod "network-metrics-daemon-rqhrw" (UID: "5dc72e56-2e79-423c-aca1-db45ca996cda") : secret "metrics-daemon-secret" not found
Jul 28 17:58:06.636609 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.636589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bcfd65e2-69dc-4593-aa86-aa6a8c84c366-original-pull-secret\") pod \"global-pull-secret-syncer-8lwbk\" (UID: \"bcfd65e2-69dc-4593-aa86-aa6a8c84c366\") " pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:58:06.725458 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.725402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:58:06.727902 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.727872    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m985w\" (UniqueName: \"kubernetes.io/projected/566d6462-3b3d-410b-b27a-34a0d1a74d15-kube-api-access-m985w\") pod \"network-check-target-tsqvb\" (UID: \"566d6462-3b3d-410b-b27a-34a0d1a74d15\") " pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:58:06.871343 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.871309    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgwbs\""
Jul 28 17:58:06.876554 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.876509    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8lwbk"
Jul 28 17:58:06.879308 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:06.879291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:58:07.010864 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.010832    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsqvb"]
Jul 28 17:58:07.014816 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:07.014788    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566d6462_3b3d_410b_b27a_34a0d1a74d15.slice/crio-4ee865d6330817dbf1d2a5fe95106584858b9e4b0c24db1d03f918ef7f0c1257 WatchSource:0}: Error finding container 4ee865d6330817dbf1d2a5fe95106584858b9e4b0c24db1d03f918ef7f0c1257: Status 404 returned error can't find the container with id 4ee865d6330817dbf1d2a5fe95106584858b9e4b0c24db1d03f918ef7f0c1257
Jul 28 17:58:07.028098 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.028075    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8lwbk"]
Jul 28 17:58:07.031629 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:07.031606    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfd65e2_69dc_4593_aa86_aa6a8c84c366.slice/crio-dffce4f9da39c01e929f83a5efc0e7d9ab702706713dd87180c1f883e2bc0354 WatchSource:0}: Error finding container dffce4f9da39c01e929f83a5efc0e7d9ab702706713dd87180c1f883e2bc0354: Status 404 returned error can't find the container with id dffce4f9da39c01e929f83a5efc0e7d9ab702706713dd87180c1f883e2bc0354
Jul 28 17:58:07.229512 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.229408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:58:07.231771 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.231744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/35236c73-366b-4c06-8832-56a0790f2282-metrics-tls\") pod \"dns-default-7k8k8\" (UID: \"35236c73-366b-4c06-8832-56a0790f2282\") " pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:58:07.240259 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.240230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lwbk" event={"ID":"bcfd65e2-69dc-4593-aa86-aa6a8c84c366","Type":"ContainerStarted","Data":"dffce4f9da39c01e929f83a5efc0e7d9ab702706713dd87180c1f883e2bc0354"}
Jul 28 17:58:07.241535 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.241512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsqvb" event={"ID":"566d6462-3b3d-410b-b27a-34a0d1a74d15","Type":"ContainerStarted","Data":"4583821fe4fc75e1623b28bdca8d436a2e45705c2480ee5e833e136ceeddbb50"}
Jul 28 17:58:07.241638 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.241550    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsqvb" event={"ID":"566d6462-3b3d-410b-b27a-34a0d1a74d15","Type":"ContainerStarted","Data":"4ee865d6330817dbf1d2a5fe95106584858b9e4b0c24db1d03f918ef7f0c1257"}
Jul 28 17:58:07.241703 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.241649    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:58:07.258319 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.258250    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tsqvb" podStartSLOduration=67.25823504 podStartE2EDuration="1m7.25823504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:07.25745894 +0000 UTC m=+66.827521795" watchObservedRunningTime="2026-07-28 17:58:07.25823504 +0000 UTC m=+66.828297849"
Jul 28 17:58:07.440424 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.440391    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5mqwj\""
Jul 28 17:58:07.449150 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.449121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:58:07.593627 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:07.593596    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7k8k8"]
Jul 28 17:58:07.596780 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:07.596751    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35236c73_366b_4c06_8832_56a0790f2282.slice/crio-989869c4400bde690af7377ad25eb96fb2f37de3fa31bd616ab6481eecca2e72 WatchSource:0}: Error finding container 989869c4400bde690af7377ad25eb96fb2f37de3fa31bd616ab6481eecca2e72: Status 404 returned error can't find the container with id 989869c4400bde690af7377ad25eb96fb2f37de3fa31bd616ab6481eecca2e72
Jul 28 17:58:08.246571 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:08.246532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7k8k8" event={"ID":"35236c73-366b-4c06-8832-56a0790f2282","Type":"ContainerStarted","Data":"989869c4400bde690af7377ad25eb96fb2f37de3fa31bd616ab6481eecca2e72"}
Jul 28 17:58:10.964840 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:10.964807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:10.965322 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:10.964913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:10.965579 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:10.965525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f98f8ecb-0659-442b-9b35-eefff9f4007d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:10.967710 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:10.967685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f98f8ecb-0659-442b-9b35-eefff9f4007d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ndmx\" (UID: \"f98f8ecb-0659-442b-9b35-eefff9f4007d\") " pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:11.101217 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.101193    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7kkv\""
Jul 28 17:58:11.109198 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.109170    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ndmx"
Jul 28 17:58:11.260999 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.260907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7k8k8" event={"ID":"35236c73-366b-4c06-8832-56a0790f2282","Type":"ContainerStarted","Data":"ba9506c4c86c347c56ba8d05d1971c67eb41647b0487d91b3cce635b332123ab"}
Jul 28 17:58:11.260999 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.260993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7k8k8" event={"ID":"35236c73-366b-4c06-8832-56a0790f2282","Type":"ContainerStarted","Data":"98cb2e0458ac2d470707b878e3458d880699e13c3e408280447692f0a540de35"}
Jul 28 17:58:11.261218 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.261016    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:58:11.262449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.262423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8lwbk" event={"ID":"bcfd65e2-69dc-4593-aa86-aa6a8c84c366","Type":"ContainerStarted","Data":"f0d0640206c7eb8798bd0aa103bd36e9afe8067cb699c57889e4fe27ea105c2d"}
Jul 28 17:58:11.266959 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.266927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ndmx"]
Jul 28 17:58:11.270061 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:11.270034    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98f8ecb_0659_442b_9b35_eefff9f4007d.slice/crio-e8a23f06e9660cba28f43193a253061b7e1fc941eb7c718c2cc06fc46386bb38 WatchSource:0}: Error finding container e8a23f06e9660cba28f43193a253061b7e1fc941eb7c718c2cc06fc46386bb38: Status 404 returned error can't find the container with id e8a23f06e9660cba28f43193a253061b7e1fc941eb7c718c2cc06fc46386bb38
Jul 28 17:58:11.285805 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.285753    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7k8k8" podStartSLOduration=32.862599984 podStartE2EDuration="36.285733447s" totalImagesPullingTime="3.423133463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:11.284825814 +0000 UTC m=+70.854888657" watchObservedRunningTime="2026-07-28 17:58:11.285733447 +0000 UTC m=+70.855796280"
Jul 28 17:58:11.301428 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:11.301377    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8lwbk" podStartSLOduration=66.30116513 podStartE2EDuration="1m10.301363207s" totalImagesPullingTime="4.000198077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:11.300289761 +0000 UTC m=+70.870352584" watchObservedRunningTime="2026-07-28 17:58:11.301363207 +0000 UTC m=+70.871426037"
Jul 28 17:58:12.269138 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:12.269101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ndmx" event={"ID":"f98f8ecb-0659-442b-9b35-eefff9f4007d","Type":"ContainerStarted","Data":"e8a23f06e9660cba28f43193a253061b7e1fc941eb7c718c2cc06fc46386bb38"}
Jul 28 17:58:16.284508 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:16.284471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ndmx" event={"ID":"f98f8ecb-0659-442b-9b35-eefff9f4007d","Type":"ContainerStarted","Data":"785e354cf173dc9757a2d97a44d1b864315ca8ace2f55c190446a58c562b6979"}
Jul 28 17:58:18.504550 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.504510    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"]
Jul 28 17:58:18.519961 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.519927    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.525144 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.525120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 17:58:18.526179 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.526151    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-x4c9b\""
Jul 28 17:58:18.526179 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.526166    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 17:58:18.526477 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.526220    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 17:58:18.526477 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.526160    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 17:58:18.543769 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.543740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"]
Jul 28 17:58:18.637889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.637623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz766\" (UniqueName: \"kubernetes.io/projected/4406b185-6a9e-42bf-9117-61f05b86b835-kube-api-access-lz766\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.637889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.637715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4406b185-6a9e-42bf-9117-61f05b86b835-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.696605 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.696573    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:58:18.738869 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.738828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz766\" (UniqueName: \"kubernetes.io/projected/4406b185-6a9e-42bf-9117-61f05b86b835-kube-api-access-lz766\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.739071 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.738895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4406b185-6a9e-42bf-9117-61f05b86b835-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.741995 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.741596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4406b185-6a9e-42bf-9117-61f05b86b835-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.756468 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.756394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8f69b6c-vm7sw"]
Jul 28 17:58:18.762096 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.762036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz766\" (UniqueName: \"kubernetes.io/projected/4406b185-6a9e-42bf-9117-61f05b86b835-kube-api-access-lz766\") pod \"managed-serviceaccount-addon-agent-77f78586dd-sh224\" (UID: \"4406b185-6a9e-42bf-9117-61f05b86b835\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.765853 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.765819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.777431 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.777401    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f69b6c-vm7sw"]
Jul 28 17:58:18.840192 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-installation-pull-secrets\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-bound-sa-token\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-tls\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-trusted-ca\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-image-registry-private-configuration\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840405 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87658cb9-e17e-40cf-8beb-10cf0a888659-ca-trust-extracted\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840602 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-certificates\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.840602 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.840499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj8j4\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-kube-api-access-xj8j4\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.842123 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.842101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"
Jul 28 17:58:18.941124 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-installation-pull-secrets\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941124 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-bound-sa-token\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941395 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-tls\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941395 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-trusted-ca\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941395 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-image-registry-private-configuration\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941395 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87658cb9-e17e-40cf-8beb-10cf0a888659-ca-trust-extracted\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941791 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-certificates\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941791 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj8j4\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-kube-api-access-xj8j4\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.941791 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.941736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87658cb9-e17e-40cf-8beb-10cf0a888659-ca-trust-extracted\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.942383 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.942333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-certificates\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.942471 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.942442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87658cb9-e17e-40cf-8beb-10cf0a888659-trusted-ca\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.944116 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.944091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-installation-pull-secrets\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.944209 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.944118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87658cb9-e17e-40cf-8beb-10cf0a888659-image-registry-private-configuration\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.944365 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.944351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-registry-tls\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.953896 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.953874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj8j4\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-kube-api-access-xj8j4\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:18.959045 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:18.959025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87658cb9-e17e-40cf-8beb-10cf0a888659-bound-sa-token\") pod \"image-registry-8f69b6c-vm7sw\" (UID: \"87658cb9-e17e-40cf-8beb-10cf0a888659\") " pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:19.078517 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:19.078477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:19.893105 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:19.893074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8f69b6c-vm7sw"]
Jul 28 17:58:19.896219 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:19.896192    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87658cb9_e17e_40cf_8beb_10cf0a888659.slice/crio-8595f395b2ef6bce4939f4e4e6060ee145290cdab25073a0255ff9adb1cc9815 WatchSource:0}: Error finding container 8595f395b2ef6bce4939f4e4e6060ee145290cdab25073a0255ff9adb1cc9815: Status 404 returned error can't find the container with id 8595f395b2ef6bce4939f4e4e6060ee145290cdab25073a0255ff9adb1cc9815
Jul 28 17:58:19.907907 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:19.907887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224"]
Jul 28 17:58:19.910177 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:19.910135    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4406b185_6a9e_42bf_9117_61f05b86b835.slice/crio-a5c6a41323e6e2f5b9ecea9ff465f9cbdca0a45bf0b853c3d8d245f1263d22be WatchSource:0}: Error finding container a5c6a41323e6e2f5b9ecea9ff465f9cbdca0a45bf0b853c3d8d245f1263d22be: Status 404 returned error can't find the container with id a5c6a41323e6e2f5b9ecea9ff465f9cbdca0a45bf0b853c3d8d245f1263d22be
Jul 28 17:58:20.304377 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.304255    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ndmx" event={"ID":"f98f8ecb-0659-442b-9b35-eefff9f4007d","Type":"ContainerStarted","Data":"2849fe98d219ef0b4317d0671ececb1d02a0125d3e286bb84e98b19f6c4bc7e3"}
Jul 28 17:58:20.306584 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.306545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224" event={"ID":"4406b185-6a9e-42bf-9117-61f05b86b835","Type":"ContainerStarted","Data":"a5c6a41323e6e2f5b9ecea9ff465f9cbdca0a45bf0b853c3d8d245f1263d22be"}
Jul 28 17:58:20.308385 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.308359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw" event={"ID":"87658cb9-e17e-40cf-8beb-10cf0a888659","Type":"ContainerStarted","Data":"d246ee21b331daeb0cd087cb454e0a136e3081272957c7784a02df05231d3d0f"}
Jul 28 17:58:20.308504 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.308401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw" event={"ID":"87658cb9-e17e-40cf-8beb-10cf0a888659","Type":"ContainerStarted","Data":"8595f395b2ef6bce4939f4e4e6060ee145290cdab25073a0255ff9adb1cc9815"}
Jul 28 17:58:20.308569 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.308499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:20.333096 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.333043    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw" podStartSLOduration=2.333026085 podStartE2EDuration="2.333026085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:20.3321664 +0000 UTC m=+79.902229233" watchObservedRunningTime="2026-07-28 17:58:20.333026085 +0000 UTC m=+79.903088915"
Jul 28 17:58:20.661056 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.661009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:58:20.664054 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.664028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff634b17-0e6a-456b-b337-2db00422219d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-t7586\" (UID: \"ff634b17-0e6a-456b-b337-2db00422219d\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:58:20.894634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.894599    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d82vv\""
Jul 28 17:58:20.902558 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:20.902527    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"
Jul 28 17:58:21.064068 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:21.062813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586"]
Jul 28 17:58:21.070884 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:21.070854    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff634b17_0e6a_456b_b337_2db00422219d.slice/crio-9a7a2878d76aa2cec175654875f3a2d778a9f1a7141858409946523988069493 WatchSource:0}: Error finding container 9a7a2878d76aa2cec175654875f3a2d778a9f1a7141858409946523988069493: Status 404 returned error can't find the container with id 9a7a2878d76aa2cec175654875f3a2d778a9f1a7141858409946523988069493
Jul 28 17:58:21.272827 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:21.272749    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7k8k8"
Jul 28 17:58:21.315517 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:21.315454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586" event={"ID":"ff634b17-0e6a-456b-b337-2db00422219d","Type":"ContainerStarted","Data":"9a7a2878d76aa2cec175654875f3a2d778a9f1a7141858409946523988069493"}
Jul 28 17:58:26.335200 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.335156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ndmx" event={"ID":"f98f8ecb-0659-442b-9b35-eefff9f4007d","Type":"ContainerStarted","Data":"1fa1adc52f4977dcf9e7989e342ecd5c08031d152cd4dca91fbc1d17cdfb0ed1"}
Jul 28 17:58:26.336790 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.336767    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586" event={"ID":"ff634b17-0e6a-456b-b337-2db00422219d","Type":"ContainerStarted","Data":"e067d06e129562a347409440a90eb6b76d9c321ef6054619a83f606c8ef17ab2"}
Jul 28 17:58:26.338179 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.338158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224" event={"ID":"4406b185-6a9e-42bf-9117-61f05b86b835","Type":"ContainerStarted","Data":"258b7ec511d19423a2d7802170aad12305f1be49907286382ceda5ca9a6e574a"}
Jul 28 17:58:26.355863 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.355810    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6ndmx" podStartSLOduration=17.41323297 podStartE2EDuration="31.355795873s" totalImagesPullingTime="13.942562903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:57:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.354562167 +0000 UTC m=+85.924624999" watchObservedRunningTime="2026-07-28 17:58:26.355795873 +0000 UTC m=+85.925858704"
Jul 28 17:58:26.369918 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.369866    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-t7586" podStartSLOduration=34.103747953 podStartE2EDuration="38.369848811s" totalImagesPullingTime="4.266100858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.369512902 +0000 UTC m=+85.939575735" watchObservedRunningTime="2026-07-28 17:58:26.369848811 +0000 UTC m=+85.939911642"
Jul 28 17:58:26.385111 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:26.385063    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77f78586dd-sh224" podStartSLOduration=2.950713765 podStartE2EDuration="8.385047692s" totalImagesPullingTime="5.434333927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:26.38365402 +0000 UTC m=+85.953716851" watchObservedRunningTime="2026-07-28 17:58:26.385047692 +0000 UTC m=+85.955110525"
Jul 28 17:58:28.702018 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:28.701988    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:35.052110 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.052075    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:58:35.057889 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.057859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.060933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.060900    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 17:58:35.061118 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061045    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 17:58:35.061196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061119    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 17:58:35.061196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061139    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 17:58:35.061196 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061142    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 17:58:35.061365 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061201    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 17:58:35.061465 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fg5x9\""
Jul 28 17:58:35.061511 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.061479    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 17:58:35.066760 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.066740    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:58:35.066869 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.066743    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 17:58:35.182686 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.182857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58mj4\" (UniqueName: \"kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.182857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.182857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.182857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.182857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.183012 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.182917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283646 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58mj4\" (UniqueName: \"kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.283837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.283835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.284642 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.284617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.284755 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.284668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.284755 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.284683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.284829 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.284749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.286153 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.286128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.286260 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.286235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.293934 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.293908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58mj4\" (UniqueName: \"kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4\") pod \"console-645c4697b5-w56v2\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") " pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.367623 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.367598    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:35.502025 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:35.501992    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:58:35.505692 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:35.505663    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474304df_c5c0_440f_a748_906106d764ec.slice/crio-a7d013721c160d083488f8998a517e616dade5a6f74ad1fdd530b3b651b49fba WatchSource:0}: Error finding container a7d013721c160d083488f8998a517e616dade5a6f74ad1fdd530b3b651b49fba: Status 404 returned error can't find the container with id a7d013721c160d083488f8998a517e616dade5a6f74ad1fdd530b3b651b49fba
Jul 28 17:58:36.375891 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:36.375850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c4697b5-w56v2" event={"ID":"474304df-c5c0-440f-a748-906106d764ec","Type":"ContainerStarted","Data":"a7d013721c160d083488f8998a517e616dade5a6f74ad1fdd530b3b651b49fba"}
Jul 28 17:58:38.250314 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:38.250283    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tsqvb"
Jul 28 17:58:39.388447 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:39.388354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c4697b5-w56v2" event={"ID":"474304df-c5c0-440f-a748-906106d764ec","Type":"ContainerStarted","Data":"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"}
Jul 28 17:58:39.421623 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:39.421571    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-645c4697b5-w56v2" podStartSLOduration=1.580722666 podStartE2EDuration="4.421556035s" totalImagesPullingTime="2.840833369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:39.420758837 +0000 UTC m=+98.990821670" watchObservedRunningTime="2026-07-28 17:58:39.421556035 +0000 UTC m=+98.991618865"
Jul 28 17:58:41.321676 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:41.321646    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8f69b6c-vm7sw"
Jul 28 17:58:43.429489 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.429447    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lzfmg"]
Jul 28 17:58:43.432441 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.432422    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.435661 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.435635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 17:58:43.436220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.436200    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j6s8v\""
Jul 28 17:58:43.436335 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.436211    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 17:58:43.436773 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.436755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 17:58:43.436853 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.436781    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 17:58:43.446014 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.445991    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"]
Jul 28 17:58:43.448220 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.448206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.451002 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.450986    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 17:58:43.451371 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.451354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 17:58:43.451446 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.451359    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 17:58:43.451484 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.451361    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fk6dr\""
Jul 28 17:58:43.452976 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.452956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453066 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.452998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-sys\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453066 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brjdg\" (UniqueName: \"kubernetes.io/projected/accd0561-b1ee-4799-a0de-6041f81b59b1-kube-api-access-brjdg\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453160 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-root\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453160 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453160 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-textfile\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453160 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-metrics-client-ca\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453310 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-wtmp\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.453310 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.453227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.477431 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.477396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"]
Jul 28 17:58:43.554596 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-sys\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554596 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-sys\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brjdg\" (UniqueName: \"kubernetes.io/projected/accd0561-b1ee-4799-a0de-6041f81b59b1-kube-api-access-brjdg\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-root\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-root\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-textfile\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-metrics-client-ca\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-wtmp\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.554834 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-wtmp\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.554976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9f678d9-85df-4585-96c1-20086735ff25-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t2pr\" (UniqueName: \"kubernetes.io/projected/a9f678d9-85df-4585-96c1-20086735ff25-kube-api-access-4t2pr\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-textfile\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:43.555184    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 17:58:43.555325 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:43.555259    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls podName:accd0561-b1ee-4799-a0de-6041f81b59b1 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:44.055236684 +0000 UTC m=+103.625299511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls") pod "node-exporter-lzfmg" (UID: "accd0561-b1ee-4799-a0de-6041f81b59b1") : secret "node-exporter-tls" not found
Jul 28 17:58:43.555687 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-metrics-client-ca\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.555687 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.555424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.557122 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.557099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.570316 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.570291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brjdg\" (UniqueName: \"kubernetes.io/projected/accd0561-b1ee-4799-a0de-6041f81b59b1-kube-api-access-brjdg\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:43.656550 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9f678d9-85df-4585-96c1-20086735ff25-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t2pr\" (UniqueName: \"kubernetes.io/projected/a9f678d9-85df-4585-96c1-20086735ff25-kube-api-access-4t2pr\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:43.656691    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 17:58:43.656717 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.656712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.656979 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:43.656787    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls podName:a9f678d9-85df-4585-96c1-20086735ff25 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:44.156760067 +0000 UTC m=+103.726822898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-tnftj" (UID: "a9f678d9-85df-4585-96c1-20086735ff25") : secret "kube-state-metrics-tls" not found
Jul 28 17:58:43.657216 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.657080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a9f678d9-85df-4585-96c1-20086735ff25-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.657399 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.657381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.657473 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.657456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a9f678d9-85df-4585-96c1-20086735ff25-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.659062 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.659040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.665931 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.665900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t2pr\" (UniqueName: \"kubernetes.io/projected/a9f678d9-85df-4585-96c1-20086735ff25-kube-api-access-4t2pr\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:43.717759 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.717649    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" podUID="207cc8b2-bbc8-4b05-b73d-d41921803c96" containerName="registry" containerID="cri-o://f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6" gracePeriod=30
Jul 28 17:58:43.968446 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:43.968386    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:44.059705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059672    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059743    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059843    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059873    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqxl6\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.059933 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059909    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.060170 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.059935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca\") pod \"207cc8b2-bbc8-4b05-b73d-d41921803c96\" (UID: \"207cc8b2-bbc8-4b05-b73d-d41921803c96\") "
Jul 28 17:58:44.060170 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.060149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:44.060924 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.060529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:58:44.061122 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.061065    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:58:44.062589 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.062522    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:58:44.062702 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.062629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:58:44.062752 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.062733    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:58:44.062843 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.062816    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:58:44.062943 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.062920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/accd0561-b1ee-4799-a0de-6041f81b59b1-node-exporter-tls\") pod \"node-exporter-lzfmg\" (UID: \"accd0561-b1ee-4799-a0de-6041f81b59b1\") " pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:44.063022 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.063005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6" (OuterVolumeSpecName: "kube-api-access-bqxl6") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "kube-api-access-bqxl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:58:44.069233 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.069204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "207cc8b2-bbc8-4b05-b73d-d41921803c96" (UID: "207cc8b2-bbc8-4b05-b73d-d41921803c96"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 17:58:44.161122 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161141    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161158    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqxl6\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-kube-api-access-bqxl6\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161172    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/207cc8b2-bbc8-4b05-b73d-d41921803c96-bound-sa-token\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161186    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-trusted-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161198    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-image-registry-private-configuration\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:44.161204    2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:44.161263    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls podName:a9f678d9-85df-4585-96c1-20086735ff25 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:45.161248695 +0000 UTC m=+104.731311507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-tnftj" (UID: "a9f678d9-85df-4585-96c1-20086735ff25") : secret "kube-state-metrics-tls" not found
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161211    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/207cc8b2-bbc8-4b05-b73d-d41921803c96-installation-pull-secrets\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161301    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/207cc8b2-bbc8-4b05-b73d-d41921803c96-ca-trust-extracted\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.161333 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.161313    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/207cc8b2-bbc8-4b05-b73d-d41921803c96-registry-certificates\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:58:44.341841 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.341802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lzfmg"
Jul 28 17:58:44.352213 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:44.352183    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaccd0561_b1ee_4799_a0de_6041f81b59b1.slice/crio-b8135f04e47dea51da3b2856e311f91fda8926416d2f86986fd0ebbf7ad01586 WatchSource:0}: Error finding container b8135f04e47dea51da3b2856e311f91fda8926416d2f86986fd0ebbf7ad01586: Status 404 returned error can't find the container with id b8135f04e47dea51da3b2856e311f91fda8926416d2f86986fd0ebbf7ad01586
Jul 28 17:58:44.406014 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.405978    2581 generic.go:356] "Generic (PLEG): container finished" podID="207cc8b2-bbc8-4b05-b73d-d41921803c96" containerID="f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6" exitCode=0
Jul 28 17:58:44.406193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.406030    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b578869dd-lg22w"
Jul 28 17:58:44.406193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.406050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" event={"ID":"207cc8b2-bbc8-4b05-b73d-d41921803c96","Type":"ContainerDied","Data":"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"}
Jul 28 17:58:44.406193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.406111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b578869dd-lg22w" event={"ID":"207cc8b2-bbc8-4b05-b73d-d41921803c96","Type":"ContainerDied","Data":"a48c65b661078fc9726a2f8f044a9ee956122b92dfe58122d315a24b4cb26352"}
Jul 28 17:58:44.406193 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.406139    2581 scope.go:117] "RemoveContainer" containerID="f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"
Jul 28 17:58:44.407445 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.407422    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzfmg" event={"ID":"accd0561-b1ee-4799-a0de-6041f81b59b1","Type":"ContainerStarted","Data":"b8135f04e47dea51da3b2856e311f91fda8926416d2f86986fd0ebbf7ad01586"}
Jul 28 17:58:44.414064 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.414046    2581 scope.go:117] "RemoveContainer" containerID="f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"
Jul 28 17:58:44.414336 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:44.414317    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6\": container with ID starting with f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6 not found: ID does not exist" containerID="f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"
Jul 28 17:58:44.414413 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.414349    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6"} err="failed to get container status \"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6\": rpc error: code = NotFound desc = could not find container \"f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6\": container with ID starting with f06a88a9436c547afb77a7408cabbbb434b1d17914443638225375420f11d0b6 not found: ID does not exist"
Jul 28 17:58:44.453465 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.453425    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:58:44.461313 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.461265    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b578869dd-lg22w"]
Jul 28 17:58:44.964544 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:44.964509    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="207cc8b2-bbc8-4b05-b73d-d41921803c96" path="/var/lib/kubelet/pods/207cc8b2-bbc8-4b05-b73d-d41921803c96/volumes"
Jul 28 17:58:45.170517 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.170483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:45.173130 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.173096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9f678d9-85df-4585-96c1-20086735ff25-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-tnftj\" (UID: \"a9f678d9-85df-4585-96c1-20086735ff25\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:45.256213 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.256186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"
Jul 28 17:58:45.368927 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.368659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:45.368927 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.368721    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:45.374080 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.374054    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:45.405910 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.405878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj"]
Jul 28 17:58:45.410020 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:45.409757    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f678d9_85df_4585_96c1_20086735ff25.slice/crio-9bc442f9d122800b93d9e2a27a814b28f80f3ef562b941b01d9d729c6c78d1c3 WatchSource:0}: Error finding container 9bc442f9d122800b93d9e2a27a814b28f80f3ef562b941b01d9d729c6c78d1c3: Status 404 returned error can't find the container with id 9bc442f9d122800b93d9e2a27a814b28f80f3ef562b941b01d9d729c6c78d1c3
Jul 28 17:58:45.416984 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.416938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzfmg" event={"ID":"accd0561-b1ee-4799-a0de-6041f81b59b1","Type":"ContainerStarted","Data":"29208cb16d9d9cf4bc9ca62458e877f025d3007cfc1e001bbf3956ef39230b76"}
Jul 28 17:58:45.421023 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:45.421000    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:58:46.422469 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:46.422434    2581 generic.go:356] "Generic (PLEG): container finished" podID="accd0561-b1ee-4799-a0de-6041f81b59b1" containerID="29208cb16d9d9cf4bc9ca62458e877f025d3007cfc1e001bbf3956ef39230b76" exitCode=0
Jul 28 17:58:46.422942 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:46.422476    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzfmg" event={"ID":"accd0561-b1ee-4799-a0de-6041f81b59b1","Type":"ContainerDied","Data":"29208cb16d9d9cf4bc9ca62458e877f025d3007cfc1e001bbf3956ef39230b76"}
Jul 28 17:58:46.424341 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:46.424312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj" event={"ID":"a9f678d9-85df-4585-96c1-20086735ff25","Type":"ContainerStarted","Data":"9bc442f9d122800b93d9e2a27a814b28f80f3ef562b941b01d9d729c6c78d1c3"}
Jul 28 17:58:47.429759 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.429726    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzfmg" event={"ID":"accd0561-b1ee-4799-a0de-6041f81b59b1","Type":"ContainerStarted","Data":"70b55a06c0b6c6c72b1da9952882946583e475125617e63428502bee7af9a96a"}
Jul 28 17:58:47.430149 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.429784    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzfmg" event={"ID":"accd0561-b1ee-4799-a0de-6041f81b59b1","Type":"ContainerStarted","Data":"ae9ddff5c005a396450e8dc3a2947f089db68fc726a832c9e2d4be5e4823f1de"}
Jul 28 17:58:47.431596 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.431564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj" event={"ID":"a9f678d9-85df-4585-96c1-20086735ff25","Type":"ContainerStarted","Data":"03bc85a762220d54b1181610ff465b7a25728b439cbe5c295d34bb3f6d420eef"}
Jul 28 17:58:47.431760 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.431607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj" event={"ID":"a9f678d9-85df-4585-96c1-20086735ff25","Type":"ContainerStarted","Data":"8111c90f1b24d212b2af68fc5fd2eae9037efddadbe49f362410d89c4fa35e19"}
Jul 28 17:58:47.431760 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.431627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj" event={"ID":"a9f678d9-85df-4585-96c1-20086735ff25","Type":"ContainerStarted","Data":"bf654781879a75e15b9b1765e8247c791f635ae9373e9981d3617e5fd2cbf384"}
Jul 28 17:58:47.460620 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.460566    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lzfmg" podStartSLOduration=3.482054543 podStartE2EDuration="4.460549778s" totalImagesPullingTime="903.921938ms" totalInitContainerRuntime="74.573297ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:47.45881438 +0000 UTC m=+107.028877210" watchObservedRunningTime="2026-07-28 17:58:47.460549778 +0000 UTC m=+107.030612609"
Jul 28 17:58:47.485902 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.485847    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-tnftj" podStartSLOduration=3.224841832 podStartE2EDuration="4.485829187s" totalImagesPullingTime="1.260987355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:47.485027685 +0000 UTC m=+107.055090628" watchObservedRunningTime="2026-07-28 17:58:47.485829187 +0000 UTC m=+107.055892018"
Jul 28 17:58:47.856943 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.856903    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"]
Jul 28 17:58:47.857183 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.857170    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207cc8b2-bbc8-4b05-b73d-d41921803c96" containerName="registry"
Jul 28 17:58:47.857229 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.857184    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="207cc8b2-bbc8-4b05-b73d-d41921803c96" containerName="registry"
Jul 28 17:58:47.857259 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.857232    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="207cc8b2-bbc8-4b05-b73d-d41921803c96" containerName="registry"
Jul 28 17:58:47.860156 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.860137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.864458 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.864424    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mrb5z\""
Jul 28 17:58:47.864582 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.864473    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 17:58:47.865009 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.864983    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 17:58:47.865133 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.865038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 17:58:47.865133 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.865051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 17:58:47.865133 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.865047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4k8phj6tavcvf\""
Jul 28 17:58:47.870917 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.870898    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"]
Jul 28 17:58:47.895221 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-client-certs\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895221 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-tls\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a11f41-e839-46a6-911f-4af0ee4650c7-audit-log\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7vtb\" (UniqueName: \"kubernetes.io/projected/40a11f41-e839-46a6-911f-4af0ee4650c7-kube-api-access-g7vtb\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-metrics-server-audit-profiles\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895466 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.895598 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.895537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-client-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:47.996332 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-client-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-client-certs\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-tls\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a11f41-e839-46a6-911f-4af0ee4650c7-audit-log\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7vtb\" (UniqueName: \"kubernetes.io/projected/40a11f41-e839-46a6-911f-4af0ee4650c7-kube-api-access-g7vtb\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-metrics-server-audit-profiles\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.996936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.997846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:47.999742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-client-ca-bundle\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.000049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40a11f41-e839-46a6-911f-4af0ee4650c7-audit-log\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.001690 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.000621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-client-certs\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.002421 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.002395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40a11f41-e839-46a6-911f-4af0ee4650c7-metrics-server-audit-profiles\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.002651 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.002630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40a11f41-e839-46a6-911f-4af0ee4650c7-secret-metrics-server-tls\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.010161 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.010137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7vtb\" (UniqueName: \"kubernetes.io/projected/40a11f41-e839-46a6-911f-4af0ee4650c7-kube-api-access-g7vtb\") pod \"metrics-server-586fc8b7fd-rp9j8\" (UID: \"40a11f41-e839-46a6-911f-4af0ee4650c7\") " pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.094679 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.094640    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"]
Jul 28 17:58:48.102064 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.102030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.104553 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.104520    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 17:58:48.105147 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.105123    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n46xk\""
Jul 28 17:58:48.110769 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.109744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"]
Jul 28 17:58:48.170023 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.169984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:58:48.198114 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.198082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm4pt\" (UniqueName: \"kubernetes.io/projected/215fc5e1-f230-4ebc-a3a6-7690653dc547-kube-api-access-qm4pt\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.198265 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.198134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.299634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.299604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4pt\" (UniqueName: \"kubernetes.io/projected/215fc5e1-f230-4ebc-a3a6-7690653dc547-kube-api-access-qm4pt\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.299801 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.299648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.299801 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:48.299766    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 17:58:48.299899 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:48.299818    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert podName:215fc5e1-f230-4ebc-a3a6-7690653dc547 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:48.799803789 +0000 UTC m=+108.369866597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-mbxq4" (UID: "215fc5e1-f230-4ebc-a3a6-7690653dc547") : secret "monitoring-plugin-cert" not found
Jul 28 17:58:48.309592 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.309560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"]
Jul 28 17:58:48.311705 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.311685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4pt\" (UniqueName: \"kubernetes.io/projected/215fc5e1-f230-4ebc-a3a6-7690653dc547-kube-api-access-qm4pt\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.312529 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:48.312504    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a11f41_e839_46a6_911f_4af0ee4650c7.slice/crio-8136180bee57a04e379275baa43f3a2760aff633d70cb9f2efe141efb662124f WatchSource:0}: Error finding container 8136180bee57a04e379275baa43f3a2760aff633d70cb9f2efe141efb662124f: Status 404 returned error can't find the container with id 8136180bee57a04e379275baa43f3a2760aff633d70cb9f2efe141efb662124f
Jul 28 17:58:48.436694 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.436598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8" event={"ID":"40a11f41-e839-46a6-911f-4af0ee4650c7","Type":"ContainerStarted","Data":"8136180bee57a04e379275baa43f3a2760aff633d70cb9f2efe141efb662124f"}
Jul 28 17:58:48.805532 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:48.805446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:48.805690 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:48.805607    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 17:58:48.805690 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:58:48.805675    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert podName:215fc5e1-f230-4ebc-a3a6-7690653dc547 nodeName:}" failed. No retries permitted until 2026-07-28 17:58:49.805657796 +0000 UTC m=+109.375720605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-mbxq4" (UID: "215fc5e1-f230-4ebc-a3a6-7690653dc547") : secret "monitoring-plugin-cert" not found
Jul 28 17:58:49.686226 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.686193    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 17:58:49.712801 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.712769    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 17:58:49.712978 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.712952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.715634 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 17:58:49.715773 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715655    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rnnd6\""
Jul 28 17:58:49.715773 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 17:58:49.715773 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715601    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 17:58:49.715773 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 17:58:49.716002 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.715986    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 17:58:49.716231 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.716089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 17:58:49.716231 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.716108    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 17:58:49.716446 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.716347    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 17:58:49.717099 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.717078    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 17:58:49.717711 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.717501    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 17:58:49.717711 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.717519    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 17:58:49.717711 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.717552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 17:58:49.717914 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.717791    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c39urtpa7gj5i\""
Jul 28 17:58:49.719969 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.719949    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 17:58:49.721139 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.721114    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 17:58:49.814699 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.814975 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.814952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp6ts\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.815393 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.815329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.818103 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.818079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/215fc5e1-f230-4ebc-a3a6-7690653dc547-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mbxq4\" (UID: \"215fc5e1-f230-4ebc-a3a6-7690653dc547\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:49.914677 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.914638    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:49.915984 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.915950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916097 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916157 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916157 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916262 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916262 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916216    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916262 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916372    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916449 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp6ts\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.916745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.916663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.917839 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.917468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.918115 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.918091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.918864 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.918437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.918864 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.918834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.919051 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.919015    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.919450 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.919393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.919654 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.919547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920302 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.919975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920302 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.919997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920302 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.920064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920535 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.920311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920679 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.920657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920802 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.920776    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.920878 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.920853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.921841 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.921799    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.922518 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.922477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.922724 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.922684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.923284 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.923242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.923545 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.923526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:49.928546 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:49.928523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp6ts\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts\") pod \"prometheus-k8s-0\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:50.026111 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:50.026070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:58:50.438315 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:50.438257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"]
Jul 28 17:58:50.441378 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:50.441349    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215fc5e1_f230_4ebc_a3a6_7690653dc547.slice/crio-dc9ea4a9cad8a21170867d8edf300ef39fb271d457a58973e9d61200d3592e9f WatchSource:0}: Error finding container dc9ea4a9cad8a21170867d8edf300ef39fb271d457a58973e9d61200d3592e9f: Status 404 returned error can't find the container with id dc9ea4a9cad8a21170867d8edf300ef39fb271d457a58973e9d61200d3592e9f
Jul 28 17:58:50.476735 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:50.476700    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 17:58:50.543016 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:58:50.542979    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078c5256_4859_44aa_8464_82bd1f50538e.slice/crio-d510c6c7b2ed67e87c3d9f657693a79c385d4677d9d1719f65cba199ceefe41b WatchSource:0}: Error finding container d510c6c7b2ed67e87c3d9f657693a79c385d4677d9d1719f65cba199ceefe41b: Status 404 returned error can't find the container with id d510c6c7b2ed67e87c3d9f657693a79c385d4677d9d1719f65cba199ceefe41b
Jul 28 17:58:51.450890 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:51.450853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8" event={"ID":"40a11f41-e839-46a6-911f-4af0ee4650c7","Type":"ContainerStarted","Data":"e1a5a66e3a201ff41cfe3b1b3bfc38daa6ed16bccea1392c52e30bcdf6c7611a"}
Jul 28 17:58:51.452620 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:51.452590    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4" event={"ID":"215fc5e1-f230-4ebc-a3a6-7690653dc547","Type":"ContainerStarted","Data":"dc9ea4a9cad8a21170867d8edf300ef39fb271d457a58973e9d61200d3592e9f"}
Jul 28 17:58:51.454170 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:51.454148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"d510c6c7b2ed67e87c3d9f657693a79c385d4677d9d1719f65cba199ceefe41b"}
Jul 28 17:58:52.459098 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:52.459059    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4" exitCode=0
Jul 28 17:58:52.459535 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:52.459145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"}
Jul 28 17:58:52.460825 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:52.460783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4" event={"ID":"215fc5e1-f230-4ebc-a3a6-7690653dc547","Type":"ContainerStarted","Data":"3d32ca4abc2c9a4f1445a9e77bd06c68d2717e5a80e1cb2e0def086c73bdd2ab"}
Jul 28 17:58:52.491226 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:52.491127    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8" podStartSLOduration=3.231496787 podStartE2EDuration="5.491109841s" totalImagesPullingTime="2.259613054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:51.471819358 +0000 UTC m=+111.041882189" watchObservedRunningTime="2026-07-28 17:58:52.491109841 +0000 UTC m=+112.061172667"
Jul 28 17:58:52.508201 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:52.508154    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4" podStartSLOduration=2.694589055 podStartE2EDuration="4.508137418s" totalImagesPullingTime="1.813548363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:52.506755232 +0000 UTC m=+112.076818054" watchObservedRunningTime="2026-07-28 17:58:52.508137418 +0000 UTC m=+112.078200249"
Jul 28 17:58:53.465640 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:53.465603    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:53.472428 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:53.472396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mbxq4"
Jul 28 17:58:57.485618 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:57.485584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"}
Jul 28 17:58:57.486043 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:57.485650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"}
Jul 28 17:58:58.492832 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:58.492801    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"}
Jul 28 17:58:59.498410 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.498329    2581 generic.go:356] "Generic (PLEG): container finished" podID="0394e7da-58ea-4122-87bb-ee65437b2e6c" containerID="f4c7bda75dd8c9e69c6c78e8a88b354e4ce1d2ea8af462d312b4af510eec77e1" exitCode=0
Jul 28 17:58:59.498837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.498400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9" event={"ID":"0394e7da-58ea-4122-87bb-ee65437b2e6c","Type":"ContainerDied","Data":"f4c7bda75dd8c9e69c6c78e8a88b354e4ce1d2ea8af462d312b4af510eec77e1"}
Jul 28 17:58:59.498837 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.498780    2581 scope.go:117] "RemoveContainer" containerID="f4c7bda75dd8c9e69c6c78e8a88b354e4ce1d2ea8af462d312b4af510eec77e1"
Jul 28 17:58:59.501818 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.501798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"}
Jul 28 17:58:59.501907 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.501853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"}
Jul 28 17:58:59.501907 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.501880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerStarted","Data":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"}
Jul 28 17:58:59.551289 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.551151    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.170047061 podStartE2EDuration="10.551125478s" totalImagesPullingTime="7.380098806s" totalInitContainerRuntime="979.611µs" isStatefulPod=false podCreationTimestamp="2026-07-28 17:58:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:58:59.549434343 +0000 UTC m=+119.119497174" watchObservedRunningTime="2026-07-28 17:58:59.551125478 +0000 UTC m=+119.121188310"
Jul 28 17:58:59.830210 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:58:59.830174    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:59:00.026377 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:00.026337    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:59:00.506906 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:00.506867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-z9sp9" event={"ID":"0394e7da-58ea-4122-87bb-ee65437b2e6c","Type":"ContainerStarted","Data":"2cd7ca4436698fb26ab58f4c97630a6e825610f5931db67f81063672d9e9a62f"}
Jul 28 17:59:08.170620 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:08.170574    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:59:08.171007 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:08.170656    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:59:10.696580 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:10.696534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:59:10.698918 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:10.698890    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dc72e56-2e79-423c-aca1-db45ca996cda-metrics-certs\") pod \"network-metrics-daemon-rqhrw\" (UID: \"5dc72e56-2e79-423c-aca1-db45ca996cda\") " pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:59:10.783063 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:10.783027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pbl4v\""
Jul 28 17:59:10.791141 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:10.791119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqhrw"
Jul 28 17:59:10.926659 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:10.926627    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqhrw"]
Jul 28 17:59:10.929424 ip-10-0-133-244 kubenswrapper[2581]: W0728 17:59:10.929386    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc72e56_2e79_423c_aca1_db45ca996cda.slice/crio-1c2aa5e472e839a297ebaa102540bec24c1d7cf0a3491484f60bb2329e42969b WatchSource:0}: Error finding container 1c2aa5e472e839a297ebaa102540bec24c1d7cf0a3491484f60bb2329e42969b: Status 404 returned error can't find the container with id 1c2aa5e472e839a297ebaa102540bec24c1d7cf0a3491484f60bb2329e42969b
Jul 28 17:59:11.552327 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:11.552246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqhrw" event={"ID":"5dc72e56-2e79-423c-aca1-db45ca996cda","Type":"ContainerStarted","Data":"1c2aa5e472e839a297ebaa102540bec24c1d7cf0a3491484f60bb2329e42969b"}
Jul 28 17:59:12.558155 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:12.558117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqhrw" event={"ID":"5dc72e56-2e79-423c-aca1-db45ca996cda","Type":"ContainerStarted","Data":"ac245bc38196a40acb1a1c7830b151ce5d52da88eef135ca463ebf9f69c63bab"}
Jul 28 17:59:12.558658 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:12.558170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqhrw" event={"ID":"5dc72e56-2e79-423c-aca1-db45ca996cda","Type":"ContainerStarted","Data":"cb1a2c9dc7dfb9c026ca4e81b27ace1dae2106d72c9a7291f7695389e0291c61"}
Jul 28 17:59:12.575683 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:12.575627    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rqhrw" podStartSLOduration=130.683233348 podStartE2EDuration="2m11.57560946s" totalImagesPullingTime="892.376112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:59:12.574616792 +0000 UTC m=+132.144679625" watchObservedRunningTime="2026-07-28 17:59:12.57560946 +0000 UTC m=+132.145672291"
Jul 28 17:59:14.568172 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:14.568084    2581 generic.go:356] "Generic (PLEG): container finished" podID="631859d9-19f5-4e56-9af5-cf1f161e40fd" containerID="fdbb69b844bf4ad0a8bdea8465845cb8522091780a33eef5668c7bb587bc5aa6" exitCode=0
Jul 28 17:59:14.568172 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:14.568128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zhnzc" event={"ID":"631859d9-19f5-4e56-9af5-cf1f161e40fd","Type":"ContainerDied","Data":"fdbb69b844bf4ad0a8bdea8465845cb8522091780a33eef5668c7bb587bc5aa6"}
Jul 28 17:59:14.568595 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:14.568460    2581 scope.go:117] "RemoveContainer" containerID="fdbb69b844bf4ad0a8bdea8465845cb8522091780a33eef5668c7bb587bc5aa6"
Jul 28 17:59:15.190507 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:15.190478    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g2hl8_8dea7d38-b4c7-49a8-80a3-c86f9e10e06b/serve-healthcheck-canary/0.log"
Jul 28 17:59:15.573825 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:15.573793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zhnzc" event={"ID":"631859d9-19f5-4e56-9af5-cf1f161e40fd","Type":"ContainerStarted","Data":"a24a87e094d78c54b85b9132a5d86d0994b3bc46f6b48dd11a43b472ad36cf0b"}
Jul 28 17:59:24.849601 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:24.849536    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-645c4697b5-w56v2" podUID="474304df-c5c0-440f-a748-906106d764ec" containerName="console" containerID="cri-o://a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306" gracePeriod=15
Jul 28 17:59:25.104774 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.104715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:59:25.223745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223711    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223745 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223787    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223839    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223945    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58mj4\" (UniqueName: \"kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.223993 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.223970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config\") pod \"474304df-c5c0-440f-a748-906106d764ec\" (UID: \"474304df-c5c0-440f-a748-906106d764ec\") "
Jul 28 17:59:25.224238 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.224157    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:59:25.224306 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.224292    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-oauth-serving-cert\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.224361 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.224307    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:59:25.224448 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.224418    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config" (OuterVolumeSpecName: "console-config") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:59:25.224482 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.224430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:59:25.226131 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.226105    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:59:25.226239 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.226127    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:59:25.226239 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.226198    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4" (OuterVolumeSpecName: "kube-api-access-58mj4") pod "474304df-c5c0-440f-a748-906106d764ec" (UID: "474304df-c5c0-440f-a748-906106d764ec"). InnerVolumeSpecName "kube-api-access-58mj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:59:25.324934 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324898    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58mj4\" (UniqueName: \"kubernetes.io/projected/474304df-c5c0-440f-a748-906106d764ec-kube-api-access-58mj4\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.324934 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324932    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-oauth-config\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.324934 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324943    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.325162 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324952    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/474304df-c5c0-440f-a748-906106d764ec-console-serving-cert\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.325162 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324961    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-trusted-ca-bundle\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.325162 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.324968    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/474304df-c5c0-440f-a748-906106d764ec-console-config\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 17:59:25.616912 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.616870    2581 generic.go:356] "Generic (PLEG): container finished" podID="474304df-c5c0-440f-a748-906106d764ec" containerID="a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306" exitCode=0
Jul 28 17:59:25.617075 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.616921    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c4697b5-w56v2"
Jul 28 17:59:25.617075 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.616959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c4697b5-w56v2" event={"ID":"474304df-c5c0-440f-a748-906106d764ec","Type":"ContainerDied","Data":"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"}
Jul 28 17:59:25.617075 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.617027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c4697b5-w56v2" event={"ID":"474304df-c5c0-440f-a748-906106d764ec","Type":"ContainerDied","Data":"a7d013721c160d083488f8998a517e616dade5a6f74ad1fdd530b3b651b49fba"}
Jul 28 17:59:25.617075 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.617057    2581 scope.go:117] "RemoveContainer" containerID="a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"
Jul 28 17:59:25.626588 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.626564    2581 scope.go:117] "RemoveContainer" containerID="a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"
Jul 28 17:59:25.626870 ip-10-0-133-244 kubenswrapper[2581]: E0728 17:59:25.626850    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306\": container with ID starting with a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306 not found: ID does not exist" containerID="a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"
Jul 28 17:59:25.626943 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.626886    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306"} err="failed to get container status \"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306\": rpc error: code = NotFound desc = could not find container \"a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306\": container with ID starting with a1743926c7141f56033c0f557d28ea13ebbf5b622d5cf49434d7b4ee4fc76306 not found: ID does not exist"
Jul 28 17:59:25.639450 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.639411    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:59:25.644245 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:25.644219    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-645c4697b5-w56v2"]
Jul 28 17:59:26.963857 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:26.963820    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474304df-c5c0-440f-a748-906106d764ec" path="/var/lib/kubelet/pods/474304df-c5c0-440f-a748-906106d764ec/volumes"
Jul 28 17:59:28.182759 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:28.181990    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:59:28.187153 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:28.187125    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-586fc8b7fd-rp9j8"
Jul 28 17:59:50.026430 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:50.026389    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:59:50.045945 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:50.045919    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 17:59:50.734017 ip-10-0-133-244 kubenswrapper[2581]: I0728 17:59:50.733979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.285718 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.285678    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.285964    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="prometheus" containerID="cri-o://229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3" gracePeriod=600
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.286020    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6" gracePeriod=600
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.286055    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-web" containerID="cri-o://e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a" gracePeriod=600
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.286069    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="config-reloader" containerID="cri-o://22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5" gracePeriod=600
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.286031    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="thanos-sidecar" containerID="cri-o://f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb" gracePeriod=600
Jul 28 18:00:08.286174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.285997    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy" containerID="cri-o://0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216" gracePeriod=600
Jul 28 18:00:08.536212 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.536151    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.601846 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601814    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.601846 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601849    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601910    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601942    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp6ts\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.601973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602043    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602170    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602197    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602213    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602222    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602299    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602331    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602388 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602403    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602440    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle\") pod \"078c5256-4859-44aa-8464-82bd1f50538e\" (UID: \"078c5256-4859-44aa-8464-82bd1f50538e\") "
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602792    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-metrics-client-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.602879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.602812    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.603401 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.603374    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:00:08.603711 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.603683    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.604710 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.604673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.605156 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.604953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out" (OuterVolumeSpecName: "config-out") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 18:00:08.605156 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.604955    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.605353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.605294    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.605440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.605411    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.605684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.605655    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.606119 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.606092    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.606740 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.606510    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:00:08.606740 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.606680    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts" (OuterVolumeSpecName: "kube-api-access-gp6ts") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "kube-api-access-gp6ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:00:08.607087 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.607041    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:00:08.607410 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.607384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.607752 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.607729    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config" (OuterVolumeSpecName: "config") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.607835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.607768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.608220 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.608203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.608357 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.608247    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.617764 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.617735    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config" (OuterVolumeSpecName: "web-config") pod "078c5256-4859-44aa-8464-82bd1f50538e" (UID: "078c5256-4859-44aa-8464-82bd1f50538e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:00:08.703333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703297    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-kube-rbac-proxy\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703327    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-config\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703337    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-tls-assets\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703347    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703365    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703374    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703382    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-web-config\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703394    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-db\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703403    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703412    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp6ts\" (UniqueName: \"kubernetes.io/projected/078c5256-4859-44aa-8464-82bd1f50538e-kube-api-access-gp6ts\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703420    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/078c5256-4859-44aa-8464-82bd1f50538e-config-out\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703429    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703437    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703445    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703455    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/078c5256-4859-44aa-8464-82bd1f50538e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703463    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703471    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-grpc-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.703594 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.703478    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/078c5256-4859-44aa-8464-82bd1f50538e-secret-metrics-client-certs\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789845    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789881    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789890    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789897    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789904    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789912    2581 generic.go:356] "Generic (PLEG): container finished" podID="078c5256-4859-44aa-8464-82bd1f50538e" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3" exitCode=0
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789920    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.789952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.789942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790097    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790207    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"}
Jul 28 18:00:08.790322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.790254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"078c5256-4859-44aa-8464-82bd1f50538e","Type":"ContainerDied","Data":"d510c6c7b2ed67e87c3d9f657693a79c385d4677d9d1719f65cba199ceefe41b"}
Jul 28 18:00:08.803763 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.803741    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.810817 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.810797    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.817372 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.817343    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.823980 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.823964    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.829424 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.829399    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:08.831391 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.831364    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.835560 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.835533    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:08.839637 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.839620    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.846636 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.846619    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.846930 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.846900    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.846988 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.846940    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.846988 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.846961    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.847233 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.847215    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.847326 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847244    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.847326 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847298    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.847548 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.847528    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.847593 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847558    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.847593 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847581    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.847808 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.847791    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.847869 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847816    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.847869 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.847836    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.848105 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.848078    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.848105 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848099    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.848183 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848113    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.848333 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.848317    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.848402 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848340    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.848402 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848359    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.848563 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:00:08.848547    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.848623 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848568    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.848623 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848587    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.848798 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848780    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.848834 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.848799    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.849026 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849009    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.849069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849026    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.849233 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849215    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.849324 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849234    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.849495 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849476    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.849548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849496    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.849700 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849682    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.849769 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849702    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.849904 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849886    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.849946 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.849904    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.850083 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850065    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.850124 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850083    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.850310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850292    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.850385 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850312    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.850511 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850496    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.850560 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850512    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.850688 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850672    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.850730 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850689    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.850865 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850850    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.850865 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.850864    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.851063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851045    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.851134 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851065    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.851328 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851312    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.851381 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851328    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.851486 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851473    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.851534 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851486    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.851683 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851663    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.851751 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851684    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.851877 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851858    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.851926 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.851878    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.852048 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852031    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.852108 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852050    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.852285 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852250    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.852328 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852289    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.852480 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852463    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.852548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852481    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.852724 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852706    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.852765 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852725    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.852924 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852908    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.852963 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.852924    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.853148 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853132    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.853213 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853149    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.853378 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853354    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.853431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853380    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.853591 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853569    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.853631 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853594    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.853795 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853774    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.853795 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853793    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.853973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853958    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.854023 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.853974    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.854174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854158    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.854174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854173    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.854394 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854374    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.854446 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854394    2581 scope.go:117] "RemoveContainer" containerID="6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"
Jul 28 18:00:08.854617 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854600    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6"} err="failed to get container status \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": rpc error: code = NotFound desc = could not find container \"6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6\": container with ID starting with 6f37a084bacb363beb829e0007ef5a55529b262ebd9983d2182ee4980eb921f6 not found: ID does not exist"
Jul 28 18:00:08.854670 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854618    2581 scope.go:117] "RemoveContainer" containerID="0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"
Jul 28 18:00:08.854839 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854822    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216"} err="failed to get container status \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": rpc error: code = NotFound desc = could not find container \"0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216\": container with ID starting with 0d5db09623bdd7bcc841563d55af1c9c293ad3b7cdf95f496acc7d79aa22c216 not found: ID does not exist"
Jul 28 18:00:08.854889 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.854839    2581 scope.go:117] "RemoveContainer" containerID="e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"
Jul 28 18:00:08.855029 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855001    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a"} err="failed to get container status \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": rpc error: code = NotFound desc = could not find container \"e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a\": container with ID starting with e6f1c5ace6cbca2a728a9eff7fae7cb86a07bcb70cd371d3d73b32aa1061426a not found: ID does not exist"
Jul 28 18:00:08.855067 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855031    2581 scope.go:117] "RemoveContainer" containerID="f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"
Jul 28 18:00:08.855216 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855200    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb"} err="failed to get container status \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": rpc error: code = NotFound desc = could not find container \"f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb\": container with ID starting with f185252dfa90501b3bbfee2b420dfc00ac12967d1bc6ebf61fb505833d1325fb not found: ID does not exist"
Jul 28 18:00:08.855252 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855216    2581 scope.go:117] "RemoveContainer" containerID="22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"
Jul 28 18:00:08.855450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855433    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5"} err="failed to get container status \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": rpc error: code = NotFound desc = could not find container \"22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5\": container with ID starting with 22da1329a718ba931b38c94eaafc036e09c9b37a159a11239bde58280dbfbdb5 not found: ID does not exist"
Jul 28 18:00:08.855506 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855451    2581 scope.go:117] "RemoveContainer" containerID="229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"
Jul 28 18:00:08.855663 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855647    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3"} err="failed to get container status \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": rpc error: code = NotFound desc = could not find container \"229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3\": container with ID starting with 229ac1706aaaf1eb437a148ba9bfae5746640127b2e254bb60adf1944c5031d3 not found: ID does not exist"
Jul 28 18:00:08.855703 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855662    2581 scope.go:117] "RemoveContainer" containerID="95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"
Jul 28 18:00:08.855858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.855842    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4"} err="failed to get container status \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": rpc error: code = NotFound desc = could not find container \"95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4\": container with ID starting with 95d068d5c3d65ffeefbe1b78b275efb015b766336415d818851173bc084253c4 not found: ID does not exist"
Jul 28 18:00:08.888941 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.888908    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:08.889241 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889229    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889244    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889253    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="init-config-reloader"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889259    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="init-config-reloader"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889283    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="prometheus"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889289    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="prometheus"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889295    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="config-reloader"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889300    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="config-reloader"
Jul 28 18:00:08.889308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889308    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="thanos-sidecar"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889314    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="thanos-sidecar"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889323    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-web"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889328    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-web"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889336    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-thanos"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889341    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-thanos"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889350    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474304df-c5c0-440f-a748-906106d764ec" containerName="console"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889355    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="474304df-c5c0-440f-a748-906106d764ec" containerName="console"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889397    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="474304df-c5c0-440f-a748-906106d764ec" containerName="console"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889405    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="config-reloader"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889411    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-web"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889418    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="thanos-sidecar"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889424    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889429    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="kube-rbac-proxy-thanos"
Jul 28 18:00:08.889544 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.889435    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="078c5256-4859-44aa-8464-82bd1f50538e" containerName="prometheus"
Jul 28 18:00:08.894850 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.894828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904257 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904689    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904692    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 18:00:08.904792 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl6wn\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-kube-api-access-tl6wn\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904852    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904918    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 18:00:08.905075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.904939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:08.905342 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.905092    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 18:00:08.905733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.905716    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rnnd6\""
Jul 28 18:00:08.905800 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.905786    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 18:00:08.909389 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909371    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 18:00:08.909389 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909383    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 18:00:08.909626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 18:00:08.909626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909621    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 18:00:08.909745 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c39urtpa7gj5i\""
Jul 28 18:00:08.909919 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.909903    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 18:00:08.911072 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.911045    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 18:00:08.911463 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.911447    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 18:00:08.914022 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.914006    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 18:00:08.918995 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.918976    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 18:00:08.932374 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.932348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:08.963385 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:08.963356    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="078c5256-4859-44aa-8464-82bd1f50538e" path="/var/lib/kubelet/pods/078c5256-4859-44aa-8464-82bd1f50538e/volumes"
Jul 28 18:00:09.006162 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.006787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.006643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl6wn\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-kube-api-access-tl6wn\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.008511 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.007148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.009923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.009394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config-out\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.009923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.009511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.009923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.009867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.009923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.009893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.011179 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.010938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.011179 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.011100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.011179 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.011172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.011445 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.011226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.011952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.011907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.012093 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.012071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.012184 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.012112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.012650 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.012616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.012833 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.012797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.013333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.013297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.013427 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.013356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-web-config\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.013721 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.013697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.014008 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.013991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.014646 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.014627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.015554 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.015535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl6wn\" (UniqueName: \"kubernetes.io/projected/4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003-kube-api-access-tl6wn\") pod \"prometheus-k8s-0\" (UID: \"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.204094 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.204053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:00:09.385356 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.385325    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 18:00:09.386461 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:00:09.386433    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4f9f9d_c1eb_4e0e_8cb5_ff8e53c33003.slice/crio-2ab509deb6354b9bb5562836493501b62172e82a64dbb3a33ef5129573afd142 WatchSource:0}: Error finding container 2ab509deb6354b9bb5562836493501b62172e82a64dbb3a33ef5129573afd142: Status 404 returned error can't find the container with id 2ab509deb6354b9bb5562836493501b62172e82a64dbb3a33ef5129573afd142
Jul 28 18:00:09.796489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.796395    2581 generic.go:356] "Generic (PLEG): container finished" podID="4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003" containerID="60d8143d607e6a80c9e1677370541cd10031d802c412288e4e59fb75af7fde67" exitCode=0
Jul 28 18:00:09.796489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.796463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerDied","Data":"60d8143d607e6a80c9e1677370541cd10031d802c412288e4e59fb75af7fde67"}
Jul 28 18:00:09.796757 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:09.796528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"2ab509deb6354b9bb5562836493501b62172e82a64dbb3a33ef5129573afd142"}
Jul 28 18:00:10.804028 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.803987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"6197aafce3e262f8105ce2b38879815e7542d1b78d24d07f8ed2ea40c56144e3"}
Jul 28 18:00:10.804489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.804111    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"9073e61ee93d733d5cbb09913ea9b81e27b6bb8f26f9ea3743f4cac3da04be4d"}
Jul 28 18:00:10.804489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.804155    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"004c07b06dff786324d34a54ff58be794ec9b9a097d887995d67701b082b82a9"}
Jul 28 18:00:10.804489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.804195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"7183114ea7be2e8fb7fa9c828b53473c3a05367341a7ae52cb7c0708548630a3"}
Jul 28 18:00:10.804489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.804231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"61a9862fe91d05aabec4ae263a2c3e9b902b69815c07d542d84703ba5f9a36d8"}
Jul 28 18:00:10.804489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.804266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003","Type":"ContainerStarted","Data":"fd65eb54b67e8acff91e1a38461dfee68d0aeedf7acb9eace65740ba4aea689b"}
Jul 28 18:00:10.839248 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:10.839194    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.820146183 podStartE2EDuration="2.839177227s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.031044ms" isStatefulPod=false podCreationTimestamp="2026-07-28 18:00:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:00:10.836841562 +0000 UTC m=+190.406904395" watchObservedRunningTime="2026-07-28 18:00:10.839177227 +0000 UTC m=+190.409240057"
Jul 28 18:00:14.204583 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:00:14.204544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:01:09.205050 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:01:09.205010    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:01:09.220551 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:01:09.220526    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:01:10.038325 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:01:10.038285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 18:02:00.814295 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:02:00.814257    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 18:04:44.238577 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.238541    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"]
Jul 28 18:04:44.241589 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.241571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245149    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245162    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q9sjc\""
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245149    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245185    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245185    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 18:04:44.245320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 18:04:44.245733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1111f638-c9a8-4f15-affa-6cddc24d3f43-manager-config\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.245733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dbf8\" (UniqueName: \"kubernetes.io/projected/1111f638-c9a8-4f15-affa-6cddc24d3f43-kube-api-access-4dbf8\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.245733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.245733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.245672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-metrics-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.257307 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.257282    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"]
Jul 28 18:04:44.346738 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.346701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1111f638-c9a8-4f15-affa-6cddc24d3f43-manager-config\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.346738 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.346748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dbf8\" (UniqueName: \"kubernetes.io/projected/1111f638-c9a8-4f15-affa-6cddc24d3f43-kube-api-access-4dbf8\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.347012 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.346777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.347012 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.346799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-metrics-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.347543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.347518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1111f638-c9a8-4f15-affa-6cddc24d3f43-manager-config\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.349318 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.349297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.349422 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.349358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1111f638-c9a8-4f15-affa-6cddc24d3f43-metrics-cert\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.356192 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.356173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dbf8\" (UniqueName: \"kubernetes.io/projected/1111f638-c9a8-4f15-affa-6cddc24d3f43-kube-api-access-4dbf8\") pod \"lws-controller-manager-7cd6475d44-q42r7\" (UID: \"1111f638-c9a8-4f15-affa-6cddc24d3f43\") " pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.551221 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.551127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:44.693343 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.693309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"]
Jul 28 18:04:44.696255 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:04:44.696229    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1111f638_c9a8_4f15_affa_6cddc24d3f43.slice/crio-dd799d88864936e0ad5e0b491a69af5556f8cc7416499e158f52fa6b8df05c74 WatchSource:0}: Error finding container dd799d88864936e0ad5e0b491a69af5556f8cc7416499e158f52fa6b8df05c74: Status 404 returned error can't find the container with id dd799d88864936e0ad5e0b491a69af5556f8cc7416499e158f52fa6b8df05c74
Jul 28 18:04:44.698016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.697999    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:04:44.814421 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:44.814333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7" event={"ID":"1111f638-c9a8-4f15-affa-6cddc24d3f43","Type":"ContainerStarted","Data":"dd799d88864936e0ad5e0b491a69af5556f8cc7416499e158f52fa6b8df05c74"}
Jul 28 18:04:48.832296 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:48.832228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7" event={"ID":"1111f638-c9a8-4f15-affa-6cddc24d3f43","Type":"ContainerStarted","Data":"2fe69209818b6a77fb66ba358f3e326a689dc99fc26cbb65d9e2fbe9b6d49dd0"}
Jul 28 18:04:48.832740 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:48.832371    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:04:48.853037 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:48.852989    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7" podStartSLOduration=1.334335141 podStartE2EDuration="4.852971497s" totalImagesPullingTime="3.518636356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:04:48.851319842 +0000 UTC m=+468.421382674" watchObservedRunningTime="2026-07-28 18:04:48.852971497 +0000 UTC m=+468.423034327"
Jul 28 18:04:53.045513 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.045474    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"]
Jul 28 18:04:53.049197 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.049174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.052835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.052813    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 18:04:53.053003 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.052872    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 18:04:53.053410 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.053395    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 18:04:53.053578 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.053562    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 18:04:53.055230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.055209    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f6wlw\""
Jul 28 18:04:53.086923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.086881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"]
Jul 28 18:04:53.110774 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.110741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.110774 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.110784    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.110964 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.110870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.110964 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.110896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twpph\" (UniqueName: \"kubernetes.io/projected/5763e90e-e15e-4151-a35d-54d3702b0143-kube-api-access-twpph\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.110964 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.110932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.211990 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.211952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.211990 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.211990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.212230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.212025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.212230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.212042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpph\" (UniqueName: \"kubernetes.io/projected/5763e90e-e15e-4151-a35d-54d3702b0143-kube-api-access-twpph\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.212230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.212069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.212485 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.212454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-manifests\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.214383 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.214354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763e90e-e15e-4151-a35d-54d3702b0143-tmp\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.214597 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.214578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.214743 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.214720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5763e90e-e15e-4151-a35d-54d3702b0143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.227550 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.227523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twpph\" (UniqueName: \"kubernetes.io/projected/5763e90e-e15e-4151-a35d-54d3702b0143-kube-api-access-twpph\") pod \"opendatahub-operator-controller-manager-5fb95c4cb9-nw82x\" (UID: \"5763e90e-e15e-4151-a35d-54d3702b0143\") " pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.359770 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.359731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:53.503016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.502895    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"]
Jul 28 18:04:53.505948 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:04:53.505920    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5763e90e_e15e_4151_a35d_54d3702b0143.slice/crio-6e1e716a6bad4aec52a6db9c4fcba36d4239435359c19c57413d47224b50296d WatchSource:0}: Error finding container 6e1e716a6bad4aec52a6db9c4fcba36d4239435359c19c57413d47224b50296d: Status 404 returned error can't find the container with id 6e1e716a6bad4aec52a6db9c4fcba36d4239435359c19c57413d47224b50296d
Jul 28 18:04:53.853335 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:53.853296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x" event={"ID":"5763e90e-e15e-4151-a35d-54d3702b0143","Type":"ContainerStarted","Data":"6e1e716a6bad4aec52a6db9c4fcba36d4239435359c19c57413d47224b50296d"}
Jul 28 18:04:56.870197 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:56.870153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x" event={"ID":"5763e90e-e15e-4151-a35d-54d3702b0143","Type":"ContainerStarted","Data":"4741b6c9555b943da19183c9cbbdd84b46ad9a799e55f3048a3d8bd70f093cf1"}
Jul 28 18:04:57.876393 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:57.876356    2581 generic.go:356] "Generic (PLEG): container finished" podID="5763e90e-e15e-4151-a35d-54d3702b0143" containerID="4741b6c9555b943da19183c9cbbdd84b46ad9a799e55f3048a3d8bd70f093cf1" exitCode=0
Jul 28 18:04:57.876778 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:57.876440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x" event={"ID":"5763e90e-e15e-4151-a35d-54d3702b0143","Type":"ContainerDied","Data":"4741b6c9555b943da19183c9cbbdd84b46ad9a799e55f3048a3d8bd70f093cf1"}
Jul 28 18:04:58.882906 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:58.882869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x" event={"ID":"5763e90e-e15e-4151-a35d-54d3702b0143","Type":"ContainerStarted","Data":"4e83fe6e967231a28498c3fda5a6bd79ef8a8e7e82db3f859881e2753a2cc4fa"}
Jul 28 18:04:58.883375 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:58.883022    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:04:58.935671 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:58.935615    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x" podStartSLOduration=1.544821862 podStartE2EDuration="5.935597691s" totalImagesPullingTime="3.366574183s" totalInitContainerRuntime="1.024201646s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:04:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:04:58.931020519 +0000 UTC m=+478.501083351" watchObservedRunningTime="2026-07-28 18:04:58.935597691 +0000 UTC m=+478.505660521"
Jul 28 18:04:59.838527 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:04:59.838491    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cd6475d44-q42r7"
Jul 28 18:05:09.890073 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:09.890032    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fb95c4cb9-nw82x"
Jul 28 18:05:24.092554 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.092521    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"]
Jul 28 18:05:24.106742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.106713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.112790 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.112762    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 18:05:24.113294 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.112763    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 18:05:24.113294 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.113187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 18:05:24.113294 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.113284    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 18:05:24.113445 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.113396    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wp8m4\""
Jul 28 18:05:24.113661 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.113637    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 18:05:24.113762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.113711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 18:05:24.196492 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196667 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196667 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196667 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196667 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196667 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.196835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.196701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjh9f\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-kube-api-access-jjh9f\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.297623 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjh9f\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-kube-api-access-jjh9f\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.297816 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.297816 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.297816 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.297816 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.298022 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.297922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.298022 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.298012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.298572 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.298538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.300169 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.300140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.300266 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.300248    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.300358 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.300331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.300395 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.300337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.311023 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.310999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjh9f\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-kube-api-access-jjh9f\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.311961 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.311942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1078f0fb-b4de-46b8-a2eb-ef18c5255fab-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-nd6xq\" (UID: \"1078f0fb-b4de-46b8-a2eb-ef18c5255fab\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.316981 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.316959    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"]
Jul 28 18:05:24.416328 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.416208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:24.624305 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.624251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"]
Jul 28 18:05:24.627211 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:05:24.627184    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1078f0fb_b4de_46b8_a2eb_ef18c5255fab.slice/crio-92f5e4f5c9cb1446e79ec13b12de258535f792866c4686ff91881591508e5195 WatchSource:0}: Error finding container 92f5e4f5c9cb1446e79ec13b12de258535f792866c4686ff91881591508e5195: Status 404 returned error can't find the container with id 92f5e4f5c9cb1446e79ec13b12de258535f792866c4686ff91881591508e5195
Jul 28 18:05:24.996879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:24.996843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq" event={"ID":"1078f0fb-b4de-46b8-a2eb-ef18c5255fab","Type":"ContainerStarted","Data":"92f5e4f5c9cb1446e79ec13b12de258535f792866c4686ff91881591508e5195"}
Jul 28 18:05:27.283288 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:27.283228    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:05:27.283542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:27.283326    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 18:05:28.019893 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:28.019849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq" event={"ID":"1078f0fb-b4de-46b8-a2eb-ef18c5255fab","Type":"ContainerStarted","Data":"63bdceacb1486da593b85af34af9c64380d1a294ed9fb2532a17e22840648f7e"}
Jul 28 18:05:28.020090 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:28.019921    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:28.021895 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:28.021858    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-nd6xq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 18:05:28.022049 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:28.021920    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq" podUID="1078f0fb-b4de-46b8-a2eb-ef18c5255fab" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 18:05:29.025605 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:29.025572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq"
Jul 28 18:05:29.101699 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:05:29.101637    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-nd6xq" podStartSLOduration=2.447890945 podStartE2EDuration="5.101610499s" totalImagesPullingTime="2.653719554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:05:28.129480186 +0000 UTC m=+507.699543017" watchObservedRunningTime="2026-07-28 18:05:29.101610499 +0000 UTC m=+508.671673332"
Jul 28 18:06:58.358984 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.358938    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"]
Jul 28 18:06:58.363023 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.362990    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:06:58.374540 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.374509    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 18:06:58.374666 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.374584    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-52s2t\""
Jul 28 18:06:58.374666 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.374522    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 18:06:58.393093 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.393060    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"]
Jul 28 18:06:58.450843 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.450803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfv5s\" (UniqueName: \"kubernetes.io/projected/1ebd22e3-a69d-4369-9c30-148d0f8142f4-kube-api-access-lfv5s\") pod \"authorino-operator-75cc8d5cb7-qps2q\" (UID: \"1ebd22e3-a69d-4369-9c30-148d0f8142f4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:06:58.551484 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.551444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfv5s\" (UniqueName: \"kubernetes.io/projected/1ebd22e3-a69d-4369-9c30-148d0f8142f4-kube-api-access-lfv5s\") pod \"authorino-operator-75cc8d5cb7-qps2q\" (UID: \"1ebd22e3-a69d-4369-9c30-148d0f8142f4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:06:58.560166 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.560132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfv5s\" (UniqueName: \"kubernetes.io/projected/1ebd22e3-a69d-4369-9c30-148d0f8142f4-kube-api-access-lfv5s\") pod \"authorino-operator-75cc8d5cb7-qps2q\" (UID: \"1ebd22e3-a69d-4369-9c30-148d0f8142f4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:06:58.675660 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.675573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:06:58.815979 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:58.815948    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"]
Jul 28 18:06:58.818964 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:06:58.818927    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebd22e3_a69d_4369_9c30_148d0f8142f4.slice/crio-40ab9a3dfcb7e5787069e7857d6810db335f9e306aded27ee8c4bd022dadd0e8 WatchSource:0}: Error finding container 40ab9a3dfcb7e5787069e7857d6810db335f9e306aded27ee8c4bd022dadd0e8: Status 404 returned error can't find the container with id 40ab9a3dfcb7e5787069e7857d6810db335f9e306aded27ee8c4bd022dadd0e8
Jul 28 18:06:59.417462 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:06:59.417424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q" event={"ID":"1ebd22e3-a69d-4369-9c30-148d0f8142f4","Type":"ContainerStarted","Data":"40ab9a3dfcb7e5787069e7857d6810db335f9e306aded27ee8c4bd022dadd0e8"}
Jul 28 18:07:01.427474 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:07:01.427438    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q" event={"ID":"1ebd22e3-a69d-4369-9c30-148d0f8142f4","Type":"ContainerStarted","Data":"0c6877bd9d292a48d262d560f504dbb60db9c920c568836a91d6a3af98bb8d81"}
Jul 28 18:07:01.427860 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:07:01.427560    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:07:12.434166 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:07:12.434130    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q"
Jul 28 18:07:12.462789 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:07:12.462739    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qps2q" podStartSLOduration=12.272268432 podStartE2EDuration="14.46272141s" totalImagesPullingTime="2.190452978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:07:01.509742219 +0000 UTC m=+601.079805050" watchObservedRunningTime="2026-07-28 18:07:12.46272141 +0000 UTC m=+612.032784240"
Jul 28 18:08:35.423476 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.423439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:35.426735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.426710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.433768 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.433742    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 18:08:35.433878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.433747    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c4ndl\""
Jul 28 18:08:35.433878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.433807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 18:08:35.444289 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.444238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:35.516574 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.516539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.516742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.516600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc8kn\" (UniqueName: \"kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.617288 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.617231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.617534 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.617358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc8kn\" (UniqueName: \"kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.620008 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.619987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.632935 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.632908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc8kn\" (UniqueName: \"kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn\") pod \"maas-controller-568857964d-jsb8f\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") " pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.736880 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.736788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:35.872123 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:35.872099    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:35.874695 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:08:35.874669    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e17cbd_b989_4ce0_9dc9_578d9f9dea22.slice/crio-555adf120c8de90c67d61aa5ce26ed1b424d4cb0bf922ab31fc5ce017304bea1 WatchSource:0}: Error finding container 555adf120c8de90c67d61aa5ce26ed1b424d4cb0bf922ab31fc5ce017304bea1: Status 404 returned error can't find the container with id 555adf120c8de90c67d61aa5ce26ed1b424d4cb0bf922ab31fc5ce017304bea1
Jul 28 18:08:36.836808 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:36.836766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568857964d-jsb8f" event={"ID":"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22","Type":"ContainerStarted","Data":"555adf120c8de90c67d61aa5ce26ed1b424d4cb0bf922ab31fc5ce017304bea1"}
Jul 28 18:08:38.847450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:38.847413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568857964d-jsb8f" event={"ID":"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22","Type":"ContainerStarted","Data":"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"}
Jul 28 18:08:38.847859 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:38.847499    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:38.869215 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:38.869164    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-568857964d-jsb8f" podStartSLOduration=1.389962613 podStartE2EDuration="3.869147618s" totalImagesPullingTime="2.479185005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:08:38.868674894 +0000 UTC m=+698.438737729" watchObservedRunningTime="2026-07-28 18:08:38.869147618 +0000 UTC m=+698.439210449"
Jul 28 18:08:49.854650 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:49.854617    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:50.482966 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.482929    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:50.483186 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.483155    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-568857964d-jsb8f" podUID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" containerName="manager" containerID="cri-o://cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248" gracePeriod=30
Jul 28 18:08:50.735532 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.735474    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:50.855830 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.855787    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert\") pod \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") "
Jul 28 18:08:50.856231 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.855872    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc8kn\" (UniqueName: \"kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn\") pod \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\" (UID: \"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22\") "
Jul 28 18:08:50.858124 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.858099    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" (UID: "c2e17cbd-b989-4ce0-9dc9-578d9f9dea22"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:08:50.858229 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.858098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn" (OuterVolumeSpecName: "kube-api-access-bc8kn") pod "c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" (UID: "c2e17cbd-b989-4ce0-9dc9-578d9f9dea22"). InnerVolumeSpecName "kube-api-access-bc8kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:08:50.899924 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.899888    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" containerID="cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248" exitCode=0
Jul 28 18:08:50.899924 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.899927    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568857964d-jsb8f"
Jul 28 18:08:50.900130 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.899971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568857964d-jsb8f" event={"ID":"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22","Type":"ContainerDied","Data":"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"}
Jul 28 18:08:50.900130 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.900017    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568857964d-jsb8f" event={"ID":"c2e17cbd-b989-4ce0-9dc9-578d9f9dea22","Type":"ContainerDied","Data":"555adf120c8de90c67d61aa5ce26ed1b424d4cb0bf922ab31fc5ce017304bea1"}
Jul 28 18:08:50.900130 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.900038    2581 scope.go:117] "RemoveContainer" containerID="cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"
Jul 28 18:08:50.911934 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.911896    2581 scope.go:117] "RemoveContainer" containerID="cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"
Jul 28 18:08:50.912609 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:08:50.912563    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248\": container with ID starting with cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248 not found: ID does not exist" containerID="cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"
Jul 28 18:08:50.912688 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.912618    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248"} err="failed to get container status \"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248\": rpc error: code = NotFound desc = could not find container \"cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248\": container with ID starting with cfe948c644cfa01bfd71004d9fe875708a9459e4a80dfa3c167fa38a8758b248 not found: ID does not exist"
Jul 28 18:08:50.928216 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.928188    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:50.934403 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.934377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-568857964d-jsb8f"]
Jul 28 18:08:50.957225 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.957199    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc8kn\" (UniqueName: \"kubernetes.io/projected/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-kube-api-access-bc8kn\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:08:50.957225 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.957224    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22-webhook-cert\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:08:50.964509 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:08:50.964482    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" path="/var/lib/kubelet/pods/c2e17cbd-b989-4ce0-9dc9-578d9f9dea22/volumes"
Jul 28 18:09:10.974777 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.974733    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"]
Jul 28 18:09:10.975415 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.975286    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" containerName="manager"
Jul 28 18:09:10.975415 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.975310    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" containerName="manager"
Jul 28 18:09:10.975545 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.975418    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e17cbd-b989-4ce0-9dc9-578d9f9dea22" containerName="manager"
Jul 28 18:09:10.979315 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.979295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:10.982170 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.982147    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 18:09:10.982347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.982327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-b4b87\""
Jul 28 18:09:10.982999 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.982981    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 18:09:10.983120 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.983044    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 18:09:10.991783 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:10.991754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"]
Jul 28 18:09:11.134332 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.134266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f22cb-2a92-4457-921f-11444f9b5235-openshift-service-ca\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.134545 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.134523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.134629 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.134595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tqqv\" (UniqueName: \"kubernetes.io/projected/c77f22cb-2a92-4457-921f-11444f9b5235-kube-api-access-9tqqv\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.235396 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.235299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f22cb-2a92-4457-921f-11444f9b5235-openshift-service-ca\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.235396 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.235390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.235657 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.235425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tqqv\" (UniqueName: \"kubernetes.io/projected/c77f22cb-2a92-4457-921f-11444f9b5235-kube-api-access-9tqqv\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.235657 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:09:11.235541    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 28 18:09:11.235657 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:09:11.235613    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls podName:c77f22cb-2a92-4457-921f-11444f9b5235 nodeName:}" failed. No retries permitted until 2026-07-28 18:09:11.735594251 +0000 UTC m=+731.305657073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls") pod "maas-api-76d85b9d6c-knnsr" (UID: "c77f22cb-2a92-4457-921f-11444f9b5235") : secret "maas-api-serving-cert" not found
Jul 28 18:09:11.235968 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.235947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c77f22cb-2a92-4457-921f-11444f9b5235-openshift-service-ca\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.246992 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.246939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tqqv\" (UniqueName: \"kubernetes.io/projected/c77f22cb-2a92-4457-921f-11444f9b5235-kube-api-access-9tqqv\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.739547 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.739506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.742199 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.742169    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c77f22cb-2a92-4457-921f-11444f9b5235-maas-api-tls\") pod \"maas-api-76d85b9d6c-knnsr\" (UID: \"c77f22cb-2a92-4457-921f-11444f9b5235\") " pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.890414 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.890374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:11.924575 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.924546    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"]
Jul 28 18:09:11.928674 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.928649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:11.953776 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.953736    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 18:09:11.953951 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.953924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qdxsb\""
Jul 28 18:09:11.971851 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:11.971820    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"]
Jul 28 18:09:12.041813 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.041723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj9ct\" (UniqueName: \"kubernetes.io/projected/4777917d-cf52-4181-a872-d7e46ff6a32a-kube-api-access-lj9ct\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.042198 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.041811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4777917d-cf52-4181-a872-d7e46ff6a32a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.072858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.072826    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"]
Jul 28 18:09:12.076550 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:09:12.076518    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77f22cb_2a92_4457_921f_11444f9b5235.slice/crio-105ce63c09121656865635ee342674c8dd9b089ea4ff669dc3d0df0cd91a0e7e WatchSource:0}: Error finding container 105ce63c09121656865635ee342674c8dd9b089ea4ff669dc3d0df0cd91a0e7e: Status 404 returned error can't find the container with id 105ce63c09121656865635ee342674c8dd9b089ea4ff669dc3d0df0cd91a0e7e
Jul 28 18:09:12.142616 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.142576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4777917d-cf52-4181-a872-d7e46ff6a32a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.142789 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.142638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj9ct\" (UniqueName: \"kubernetes.io/projected/4777917d-cf52-4181-a872-d7e46ff6a32a-kube-api-access-lj9ct\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.143426 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.143407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4777917d-cf52-4181-a872-d7e46ff6a32a-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.152234 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.152203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj9ct\" (UniqueName: \"kubernetes.io/projected/4777917d-cf52-4181-a872-d7e46ff6a32a-kube-api-access-lj9ct\") pod \"payload-pre-processing-687dbb6cf6-vs95b\" (UID: \"4777917d-cf52-4181-a872-d7e46ff6a32a\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.239258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.239217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:12.382859 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.382811    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"]
Jul 28 18:09:12.385040 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:09:12.385011    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4777917d_cf52_4181_a872_d7e46ff6a32a.slice/crio-caba7376830c87c64ad8c55b1bdfaa064eb21b9ff28f805b02411b9fe06da8d4 WatchSource:0}: Error finding container caba7376830c87c64ad8c55b1bdfaa064eb21b9ff28f805b02411b9fe06da8d4: Status 404 returned error can't find the container with id caba7376830c87c64ad8c55b1bdfaa064eb21b9ff28f805b02411b9fe06da8d4
Jul 28 18:09:12.998128 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.998089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr" event={"ID":"c77f22cb-2a92-4457-921f-11444f9b5235","Type":"ContainerStarted","Data":"105ce63c09121656865635ee342674c8dd9b089ea4ff669dc3d0df0cd91a0e7e"}
Jul 28 18:09:12.999938 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:12.999903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b" event={"ID":"4777917d-cf52-4181-a872-d7e46ff6a32a","Type":"ContainerStarted","Data":"caba7376830c87c64ad8c55b1bdfaa064eb21b9ff28f805b02411b9fe06da8d4"}
Jul 28 18:09:15.015737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.015633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b" event={"ID":"4777917d-cf52-4181-a872-d7e46ff6a32a","Type":"ContainerStarted","Data":"218b9d00025beca28e73799e20626dfd1237107bd19feaced7e99673d9be4d74"}
Jul 28 18:09:15.015737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.015729    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:15.017159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.017135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr" event={"ID":"c77f22cb-2a92-4457-921f-11444f9b5235","Type":"ContainerStarted","Data":"b46a79d0e7e7426c0973159774efbc8059997f4863d4fea5a14d7f7f2f3245e9"}
Jul 28 18:09:15.017316 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.017262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:15.035511 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.035460    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b" podStartSLOduration=1.628886696 podStartE2EDuration="4.035442835s" totalImagesPullingTime="2.406556139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:15.034411126 +0000 UTC m=+734.604473953" watchObservedRunningTime="2026-07-28 18:09:15.035442835 +0000 UTC m=+734.605505666"
Jul 28 18:09:15.053465 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:15.053419    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr" podStartSLOduration=2.386876231 podStartE2EDuration="5.053401325s" totalImagesPullingTime="2.666525094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:15.051732405 +0000 UTC m=+734.621795237" watchObservedRunningTime="2026-07-28 18:09:15.053401325 +0000 UTC m=+734.623464156"
Jul 28 18:09:21.028550 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:21.028519    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76d85b9d6c-knnsr"
Jul 28 18:09:26.024004 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:26.023970    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vs95b"
Jul 28 18:09:37.382862 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.382823    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"]
Jul 28 18:09:37.386137 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.386119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.388880 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.388854    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 18:09:37.389000 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.388906    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 18:09:37.389681 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.389668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jk58p\""
Jul 28 18:09:37.389902 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.389889    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 18:09:37.401502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.401475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"]
Jul 28 18:09:37.454629 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc8x8\" (UniqueName: \"kubernetes.io/projected/a3e580e0-0dcf-4079-b634-4d0e6714d026-kube-api-access-lc8x8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.454780 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.454780 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.454780 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.454916 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e580e0-0dcf-4079-b634-4d0e6714d026-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.454916 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.454857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555942 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e580e0-0dcf-4079-b634-4d0e6714d026-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555942 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555942 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8x8\" (UniqueName: \"kubernetes.io/projected/a3e580e0-0dcf-4079-b634-4d0e6714d026-kube-api-access-lc8x8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.555942 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.555829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.556143 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.556117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.556207 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.556161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.556207 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.556173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.558054 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.558028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e580e0-0dcf-4079-b634-4d0e6714d026-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.558303 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.558266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e580e0-0dcf-4079-b634-4d0e6714d026-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.563809 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.563780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc8x8\" (UniqueName: \"kubernetes.io/projected/a3e580e0-0dcf-4079-b634-4d0e6714d026-kube-api-access-lc8x8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br\" (UID: \"a3e580e0-0dcf-4079-b634-4d0e6714d026\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.696835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.696755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:37.830863 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:37.830765    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"]
Jul 28 18:09:37.833561 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:09:37.833529    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e580e0_0dcf_4079_b634_4d0e6714d026.slice/crio-5e7b7fd6cc43795f1948d223f92e9f6dd1a09b2d411f40216da82637903052c5 WatchSource:0}: Error finding container 5e7b7fd6cc43795f1948d223f92e9f6dd1a09b2d411f40216da82637903052c5: Status 404 returned error can't find the container with id 5e7b7fd6cc43795f1948d223f92e9f6dd1a09b2d411f40216da82637903052c5
Jul 28 18:09:38.121384 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:38.121351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br" event={"ID":"a3e580e0-0dcf-4079-b634-4d0e6714d026","Type":"ContainerStarted","Data":"5e7b7fd6cc43795f1948d223f92e9f6dd1a09b2d411f40216da82637903052c5"}
Jul 28 18:09:39.127888 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:39.127850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br" event={"ID":"a3e580e0-0dcf-4079-b634-4d0e6714d026","Type":"ContainerStarted","Data":"70c61ae9122f698b23080454acc85409fa88b7adc3c52c8d67e7728401e421fd"}
Jul 28 18:09:39.147115 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:39.147062    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br" podStartSLOduration=0.973695748 podStartE2EDuration="2.147046317s" totalImagesPullingTime="1.173350569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:09:39.1458968 +0000 UTC m=+758.715959633" watchObservedRunningTime="2026-07-28 18:09:39.147046317 +0000 UTC m=+758.717109175"
Jul 28 18:09:47.697572 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:47.697505    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:47.698108 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:47.697609    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:47.710313 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:47.710262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:09:48.177704 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:09:48.177676    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br"
Jul 28 18:10:17.492434 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.492401    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"]
Jul 28 18:10:17.496108 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.496090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.498579 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.498558    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 18:10:17.510562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.510536    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"]
Jul 28 18:10:17.522649 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.522615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.522861 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.522748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.522861 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.522800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2jf\" (UniqueName: \"kubernetes.io/projected/d5eb9f97-0841-4598-a2bb-5f2e5788760c-kube-api-access-nc2jf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.522989 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.522935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.522989 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.522984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.523161 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.523020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.623659 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.623864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.623864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.623864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.624011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2jf\" (UniqueName: \"kubernetes.io/projected/d5eb9f97-0841-4598-a2bb-5f2e5788760c-kube-api-access-nc2jf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.624011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.623995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.624116 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.624041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.624178 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.624122    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.624232 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.624189    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.625896 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.625869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5eb9f97-0841-4598-a2bb-5f2e5788760c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.626240 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.626222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5eb9f97-0841-4598-a2bb-5f2e5788760c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.632098 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.632079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2jf\" (UniqueName: \"kubernetes.io/projected/d5eb9f97-0841-4598-a2bb-5f2e5788760c-kube-api-access-nc2jf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5\" (UID: \"d5eb9f97-0841-4598-a2bb-5f2e5788760c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.806527 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.806432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:17.939719 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.939694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"]
Jul 28 18:10:17.941973 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:10:17.941945    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5eb9f97_0841_4598_a2bb_5f2e5788760c.slice/crio-e71ae4157cfa9cd09019ac1f697834361283a75b790694475b68b2f1d59262f5 WatchSource:0}: Error finding container e71ae4157cfa9cd09019ac1f697834361283a75b790694475b68b2f1d59262f5: Status 404 returned error can't find the container with id e71ae4157cfa9cd09019ac1f697834361283a75b790694475b68b2f1d59262f5
Jul 28 18:10:17.943262 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:17.943241    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:10:18.298919 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:18.298883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5" event={"ID":"d5eb9f97-0841-4598-a2bb-5f2e5788760c","Type":"ContainerStarted","Data":"e71ae4157cfa9cd09019ac1f697834361283a75b790694475b68b2f1d59262f5"}
Jul 28 18:10:19.304915 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:19.304873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5" event={"ID":"d5eb9f97-0841-4598-a2bb-5f2e5788760c","Type":"ContainerStarted","Data":"cc729aab17b5c4ae2c36ae587b914f98502b0866604951b61cd1d8db4056a24f"}
Jul 28 18:10:19.327484 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:19.327427    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5" podStartSLOduration=1.932521382 podStartE2EDuration="2.327409153s" totalImagesPullingTime="394.887771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:10:19.32620491 +0000 UTC m=+798.896267740" watchObservedRunningTime="2026-07-28 18:10:19.327409153 +0000 UTC m=+798.897472024"
Jul 28 18:10:27.806627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:27.806529    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:27.806627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:27.806600    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:27.819157 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:27.819112    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:10:28.355858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:10:28.355822    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5"
Jul 28 18:13:17.404830 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.404796    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:17.408098 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.408080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.411250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.411220    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 18:13:17.411410 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.411247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 18:13:17.411410 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.411253    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c4ndl\""
Jul 28 18:13:17.417640 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.417619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:17.451420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.451386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.451593 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.451456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg5qf\" (UniqueName: \"kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.552780 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.552738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.552957 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.552812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg5qf\" (UniqueName: \"kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.555406 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.555378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.561761 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.561730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg5qf\" (UniqueName: \"kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf\") pod \"maas-controller-5655468897-fwz55\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") " pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.719897 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.719811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:17.858494 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:17.858468    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:17.861138 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:13:17.861107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda01c5ac_3fec_4c32_bbff_47d0f085cbac.slice/crio-d562edc53c36a1aaef03b724bdf1588db960d5e5ea9a6421c2be72e25cb5e035 WatchSource:0}: Error finding container d562edc53c36a1aaef03b724bdf1588db960d5e5ea9a6421c2be72e25cb5e035: Status 404 returned error can't find the container with id d562edc53c36a1aaef03b724bdf1588db960d5e5ea9a6421c2be72e25cb5e035
Jul 28 18:13:18.096869 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:18.096832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-fwz55" event={"ID":"da01c5ac-3fec-4c32-bbff-47d0f085cbac","Type":"ContainerStarted","Data":"d562edc53c36a1aaef03b724bdf1588db960d5e5ea9a6421c2be72e25cb5e035"}
Jul 28 18:13:19.102592 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:19.102553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-fwz55" event={"ID":"da01c5ac-3fec-4c32-bbff-47d0f085cbac","Type":"ContainerStarted","Data":"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"}
Jul 28 18:13:19.103081 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:19.102639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:19.128355 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:19.128295    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5655468897-fwz55" podStartSLOduration=1.5652386169999999 podStartE2EDuration="2.128258547s" totalImagesPullingTime="563.01993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:19.127003564 +0000 UTC m=+978.697066396" watchObservedRunningTime="2026-07-28 18:13:19.128258547 +0000 UTC m=+978.698321378"
Jul 28 18:13:30.109403 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:30.109373    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:34.778973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:34.778921    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:34.779416 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:34.779188    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5655468897-fwz55" podUID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" containerName="manager" containerID="cri-o://dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2" gracePeriod=30
Jul 28 18:13:35.029949 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.029886    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:35.110577 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.110544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert\") pod \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") "
Jul 28 18:13:35.110744 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.110614    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg5qf\" (UniqueName: \"kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf\") pod \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\" (UID: \"da01c5ac-3fec-4c32-bbff-47d0f085cbac\") "
Jul 28 18:13:35.112653 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.112614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "da01c5ac-3fec-4c32-bbff-47d0f085cbac" (UID: "da01c5ac-3fec-4c32-bbff-47d0f085cbac"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:13:35.112653 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.112648    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf" (OuterVolumeSpecName: "kube-api-access-pg5qf") pod "da01c5ac-3fec-4c32-bbff-47d0f085cbac" (UID: "da01c5ac-3fec-4c32-bbff-47d0f085cbac"). InnerVolumeSpecName "kube-api-access-pg5qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:13:35.173078 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.173038    2581 generic.go:356] "Generic (PLEG): container finished" podID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" containerID="dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2" exitCode=0
Jul 28 18:13:35.173297 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.173110    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-fwz55"
Jul 28 18:13:35.173297 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.173130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-fwz55" event={"ID":"da01c5ac-3fec-4c32-bbff-47d0f085cbac","Type":"ContainerDied","Data":"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"}
Jul 28 18:13:35.173297 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.173222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-fwz55" event={"ID":"da01c5ac-3fec-4c32-bbff-47d0f085cbac","Type":"ContainerDied","Data":"d562edc53c36a1aaef03b724bdf1588db960d5e5ea9a6421c2be72e25cb5e035"}
Jul 28 18:13:35.173297 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.173252    2581 scope.go:117] "RemoveContainer" containerID="dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"
Jul 28 18:13:35.182362 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.182340    2581 scope.go:117] "RemoveContainer" containerID="dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"
Jul 28 18:13:35.182616 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:13:35.182599    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2\": container with ID starting with dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2 not found: ID does not exist" containerID="dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"
Jul 28 18:13:35.182689 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.182623    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2"} err="failed to get container status \"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2\": rpc error: code = NotFound desc = could not find container \"dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2\": container with ID starting with dcc6908840052685dd1a6ed4f4d4e4a0ab2d65df44e81f162ab70122b0dbbec2 not found: ID does not exist"
Jul 28 18:13:35.198999 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.198967    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:35.206260 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.206232    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5655468897-fwz55"]
Jul 28 18:13:35.211670 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.211648    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da01c5ac-3fec-4c32-bbff-47d0f085cbac-webhook-cert\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:13:35.211744 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:35.211673    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg5qf\" (UniqueName: \"kubernetes.io/projected/da01c5ac-3fec-4c32-bbff-47d0f085cbac-kube-api-access-pg5qf\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:13:36.033304 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.033257    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5655468897-x4bbh"]
Jul 28 18:13:36.035831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.033825    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" containerName="manager"
Jul 28 18:13:36.035831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.033841    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" containerName="manager"
Jul 28 18:13:36.035831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.033896    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" containerName="manager"
Jul 28 18:13:36.036879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.036861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.039827 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.039798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 18:13:36.039827 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.039820    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c4ndl\""
Jul 28 18:13:36.039985 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.039830    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 18:13:36.049241 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.049215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-x4bbh"]
Jul 28 18:13:36.122217 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.122174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7h4t\" (UniqueName: \"kubernetes.io/projected/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-kube-api-access-z7h4t\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.122431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.122250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-webhook-cert\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.223229 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.223189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7h4t\" (UniqueName: \"kubernetes.io/projected/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-kube-api-access-z7h4t\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.223459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.223254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-webhook-cert\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.225685 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.225658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-webhook-cert\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.232962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.232938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7h4t\" (UniqueName: \"kubernetes.io/projected/2dc9ffb5-2f64-4c1f-b551-6b71479c5292-kube-api-access-z7h4t\") pod \"maas-controller-5655468897-x4bbh\" (UID: \"2dc9ffb5-2f64-4c1f-b551-6b71479c5292\") " pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.348230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.348192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:36.493359 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.493325    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5655468897-x4bbh"]
Jul 28 18:13:36.499934 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:13:36.499903    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc9ffb5_2f64_4c1f_b551_6b71479c5292.slice/crio-0ee3a05f289115d4baf1b05845c274d1317536470b972ad3c573f061263f6c2a WatchSource:0}: Error finding container 0ee3a05f289115d4baf1b05845c274d1317536470b972ad3c573f061263f6c2a: Status 404 returned error can't find the container with id 0ee3a05f289115d4baf1b05845c274d1317536470b972ad3c573f061263f6c2a
Jul 28 18:13:36.965045 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:36.965010    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da01c5ac-3fec-4c32-bbff-47d0f085cbac" path="/var/lib/kubelet/pods/da01c5ac-3fec-4c32-bbff-47d0f085cbac/volumes"
Jul 28 18:13:37.186909 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:37.186874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-x4bbh" event={"ID":"2dc9ffb5-2f64-4c1f-b551-6b71479c5292","Type":"ContainerStarted","Data":"d9955f37a0373df5b2b4b15ca4d9295114fb8b3759ce7e58c125ef6405c42bf2"}
Jul 28 18:13:37.187356 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:37.186927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5655468897-x4bbh" event={"ID":"2dc9ffb5-2f64-4c1f-b551-6b71479c5292","Type":"ContainerStarted","Data":"0ee3a05f289115d4baf1b05845c274d1317536470b972ad3c573f061263f6c2a"}
Jul 28 18:13:37.187356 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:37.186971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:13:37.205871 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:37.205813    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5655468897-x4bbh" podStartSLOduration=0.68054299 podStartE2EDuration="1.205795802s" totalImagesPullingTime="525.252812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:13:37.204490651 +0000 UTC m=+996.774553480" watchObservedRunningTime="2026-07-28 18:13:37.205795802 +0000 UTC m=+996.775858633"
Jul 28 18:13:48.194350 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:13:48.194316    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5655468897-x4bbh"
Jul 28 18:27:15.090742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.090709    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:15.094203 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.094181    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.096512 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.096480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0ab65c1c-serving-cert\""
Jul 28 18:27:15.105775 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.105745    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:15.174354 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.174318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.174538 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.174436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddf6p\" (UniqueName: \"kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.174538 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.174483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.275542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.275510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.275720 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.275564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.275720 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.275626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddf6p\" (UniqueName: \"kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.275797 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:15.275732    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-serving-cert: secret "maas-api-e2e-ait-0ab65c1c-serving-cert" not found
Jul 28 18:27:15.275831 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:15.275804    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls podName:da983ecc-4684-4205-a153-8ff7223f7a6f nodeName:}" failed. No retries permitted until 2026-07-28 18:27:15.775788003 +0000 UTC m=+1815.345850812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls") pod "maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" (UID: "da983ecc-4684-4205-a153-8ff7223f7a6f") : secret "maas-api-e2e-ait-0ab65c1c-serving-cert" not found
Jul 28 18:27:15.276204 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.276183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.286784 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.286761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddf6p\" (UniqueName: \"kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.780330 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:15.780293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:15.780521 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:15.780484    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-serving-cert: secret "maas-api-e2e-ait-0ab65c1c-serving-cert" not found
Jul 28 18:27:15.780593 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:15.780580    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls podName:da983ecc-4684-4205-a153-8ff7223f7a6f nodeName:}" failed. No retries permitted until 2026-07-28 18:27:16.780559671 +0000 UTC m=+1816.350622503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls") pod "maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" (UID: "da983ecc-4684-4205-a153-8ff7223f7a6f") : secret "maas-api-e2e-ait-0ab65c1c-serving-cert" not found
Jul 28 18:27:16.791553 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:16.791513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:16.794512 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:16.794485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:16.907599 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:16.907557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:17.288517 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:17.288479    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:17.299396 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:27:17.299365    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda983ecc_4684_4205_a153_8ff7223f7a6f.slice/crio-917338c4f97ed7a9c942bb37ba21d04f483dcc8643ae0f4aea3e253134fdbe6a WatchSource:0}: Error finding container 917338c4f97ed7a9c942bb37ba21d04f483dcc8643ae0f4aea3e253134fdbe6a: Status 404 returned error can't find the container with id 917338c4f97ed7a9c942bb37ba21d04f483dcc8643ae0f4aea3e253134fdbe6a
Jul 28 18:27:17.301069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:17.301049    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:27:17.923040 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:17.923004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" event={"ID":"da983ecc-4684-4205-a153-8ff7223f7a6f","Type":"ContainerStarted","Data":"917338c4f97ed7a9c942bb37ba21d04f483dcc8643ae0f4aea3e253134fdbe6a"}
Jul 28 18:27:18.929620 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:18.929581    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" event={"ID":"da983ecc-4684-4205-a153-8ff7223f7a6f","Type":"ContainerStarted","Data":"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"}
Jul 28 18:27:18.930061 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:18.929743    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:18.947252 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:18.947198    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" podStartSLOduration=3.283017613 podStartE2EDuration="3.947179328s" totalImagesPullingTime="664.161715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:18.946138138 +0000 UTC m=+1818.516200969" watchObservedRunningTime="2026-07-28 18:27:18.947179328 +0000 UTC m=+1818.517242159"
Jul 28 18:27:24.940439 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:24.940408    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:45.636036 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:45.635995    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:45.637151 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:45.636689    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" podUID="da983ecc-4684-4205-a153-8ff7223f7a6f" containerName="maas-api" containerID="cri-o://1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3" gracePeriod=30
Jul 28 18:27:45.886685 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:45.886625    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:46.060113 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.060080    2581 generic.go:356] "Generic (PLEG): container finished" podID="da983ecc-4684-4205-a153-8ff7223f7a6f" containerID="1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3" exitCode=0
Jul 28 18:27:46.060258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.060126    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"
Jul 28 18:27:46.060258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.060160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" event={"ID":"da983ecc-4684-4205-a153-8ff7223f7a6f","Type":"ContainerDied","Data":"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"}
Jul 28 18:27:46.060258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.060212    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n" event={"ID":"da983ecc-4684-4205-a153-8ff7223f7a6f","Type":"ContainerDied","Data":"917338c4f97ed7a9c942bb37ba21d04f483dcc8643ae0f4aea3e253134fdbe6a"}
Jul 28 18:27:46.060258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.060234    2581 scope.go:117] "RemoveContainer" containerID="1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"
Jul 28 18:27:46.064258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.064227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") pod \"da983ecc-4684-4205-a153-8ff7223f7a6f\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") "
Jul 28 18:27:46.064418 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.064297    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddf6p\" (UniqueName: \"kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p\") pod \"da983ecc-4684-4205-a153-8ff7223f7a6f\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") "
Jul 28 18:27:46.064418 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.064404    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca\") pod \"da983ecc-4684-4205-a153-8ff7223f7a6f\" (UID: \"da983ecc-4684-4205-a153-8ff7223f7a6f\") "
Jul 28 18:27:46.064734 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.064709    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da983ecc-4684-4205-a153-8ff7223f7a6f" (UID: "da983ecc-4684-4205-a153-8ff7223f7a6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:27:46.066542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.066518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da983ecc-4684-4205-a153-8ff7223f7a6f" (UID: "da983ecc-4684-4205-a153-8ff7223f7a6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:27:46.066850 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.066829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p" (OuterVolumeSpecName: "kube-api-access-ddf6p") pod "da983ecc-4684-4205-a153-8ff7223f7a6f" (UID: "da983ecc-4684-4205-a153-8ff7223f7a6f"). InnerVolumeSpecName "kube-api-access-ddf6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:27:46.069460 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.069441    2581 scope.go:117] "RemoveContainer" containerID="1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"
Jul 28 18:27:46.069756 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:46.069731    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3\": container with ID starting with 1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3 not found: ID does not exist" containerID="1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"
Jul 28 18:27:46.069822 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.069769    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3"} err="failed to get container status \"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3\": rpc error: code = NotFound desc = could not find container \"1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3\": container with ID starting with 1d1aef38a599a4ef18f4ff6df393f8d2136f000dce1920ff9ecf8e5ab3ab6cb3 not found: ID does not exist"
Jul 28 18:27:46.165574 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.165485    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddf6p\" (UniqueName: \"kubernetes.io/projected/da983ecc-4684-4205-a153-8ff7223f7a6f-kube-api-access-ddf6p\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:27:46.165574 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.165518    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da983ecc-4684-4205-a153-8ff7223f7a6f-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:27:46.165574 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.165533    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da983ecc-4684-4205-a153-8ff7223f7a6f-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:27:46.381453 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.381419    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:46.385420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.385396    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ab65c1c-77b5cbb5d7-59p4n"]
Jul 28 18:27:46.964290 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:46.964227    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da983ecc-4684-4205-a153-8ff7223f7a6f" path="/var/lib/kubelet/pods/da983ecc-4684-4205-a153-8ff7223f7a6f/volumes"
Jul 28 18:27:49.681073 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.681032    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:27:49.681664 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.681643    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da983ecc-4684-4205-a153-8ff7223f7a6f" containerName="maas-api"
Jul 28 18:27:49.681778 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.681667    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="da983ecc-4684-4205-a153-8ff7223f7a6f" containerName="maas-api"
Jul 28 18:27:49.681835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.681785    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="da983ecc-4684-4205-a153-8ff7223f7a6f" containerName="maas-api"
Jul 28 18:27:49.686557 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.686534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.689128 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.689104    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-3adc829c-serving-cert\""
Jul 28 18:27:49.695923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.695893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.696037 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.695943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.696104 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.696060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92btq\" (UniqueName: \"kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.696623 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.696572    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:27:49.796546 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.796509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92btq\" (UniqueName: \"kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.796737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.796583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.796737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.796613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.796737 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:49.796725    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-serving-cert: secret "maas-api-e2e-migrate-3adc829c-serving-cert" not found
Jul 28 18:27:49.796902 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:27:49.796794    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls podName:8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe nodeName:}" failed. No retries permitted until 2026-07-28 18:27:50.296776306 +0000 UTC m=+1849.866839142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls") pod "maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" (UID: "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe") : secret "maas-api-e2e-migrate-3adc829c-serving-cert" not found
Jul 28 18:27:49.797194 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.797179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:49.807687 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:49.807660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92btq\" (UniqueName: \"kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:50.299395 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.299354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:50.302429 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.302399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") pod \"maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:50.569737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.569650    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:27:50.573481 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.573463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.576013 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.575988    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-3adc829c\""
Jul 28 18:27:50.585344 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.585315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:27:50.589105 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.589081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:27:50.589227 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.589175    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.591869 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.591847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-3adc829c-dockercfg-x9mm6\""
Jul 28 18:27:50.600232 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.600024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:50.601006 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.600724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.601006 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.600778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8l95\" (UniqueName: \"kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.601006 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.600865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g77w9\" (UniqueName: \"kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.601265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.601192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.603477 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.603371    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:27:50.702858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.702802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.702858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.702906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8l95\" (UniqueName: \"kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.702858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.703078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g77w9\" (UniqueName: \"kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.702858 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.703204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.704198 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.703948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.704198 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.704116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.714572 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.714541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g77w9\" (UniqueName: \"kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9\") pod \"payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.726194 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.725583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8l95\" (UniqueName: \"kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95\") pod \"payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") " pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:50.759776 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.759742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:27:50.763728 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:27:50.763703    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8edfcc8e_2d5a_46d5_b8fc_356e6e6a62fe.slice/crio-681ead7213bac5ccb7a4ad1a0019d4281c941091646129b885aea15785643739 WatchSource:0}: Error finding container 681ead7213bac5ccb7a4ad1a0019d4281c941091646129b885aea15785643739: Status 404 returned error can't find the container with id 681ead7213bac5ccb7a4ad1a0019d4281c941091646129b885aea15785643739
Jul 28 18:27:50.886374 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.886332    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:50.900107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:50.900071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:51.082907 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:51.082881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:27:51.084831 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:27:51.084802    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785949fd_0f51_47b1_b6bd_33d051041eb8.slice/crio-202c108934e4f628edaca9ea79c0b58ded4cabaa189ab62e94d86a57a652442d WatchSource:0}: Error finding container 202c108934e4f628edaca9ea79c0b58ded4cabaa189ab62e94d86a57a652442d: Status 404 returned error can't find the container with id 202c108934e4f628edaca9ea79c0b58ded4cabaa189ab62e94d86a57a652442d
Jul 28 18:27:51.086440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:51.086411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" event={"ID":"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe","Type":"ContainerStarted","Data":"681ead7213bac5ccb7a4ad1a0019d4281c941091646129b885aea15785643739"}
Jul 28 18:27:51.257009 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:51.256980    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:27:51.260022 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:27:51.259995    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0f2a82_83a1_4faf_9ceb_3bf6f7114d5f.slice/crio-4dc126d2e16c39b7530e330bc095da50093cd8a27158d8b1e02078fdd1c3c1a5 WatchSource:0}: Error finding container 4dc126d2e16c39b7530e330bc095da50093cd8a27158d8b1e02078fdd1c3c1a5: Status 404 returned error can't find the container with id 4dc126d2e16c39b7530e330bc095da50093cd8a27158d8b1e02078fdd1c3c1a5
Jul 28 18:27:52.093006 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.092962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" event={"ID":"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe","Type":"ContainerStarted","Data":"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"}
Jul 28 18:27:52.093493 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.093036    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:27:52.094872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.094844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" event={"ID":"785949fd-0f51-47b1-b6bd-33d051041eb8","Type":"ContainerStarted","Data":"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"}
Jul 28 18:27:52.095016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.094891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" event={"ID":"785949fd-0f51-47b1-b6bd-33d051041eb8","Type":"ContainerStarted","Data":"202c108934e4f628edaca9ea79c0b58ded4cabaa189ab62e94d86a57a652442d"}
Jul 28 18:27:52.095016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.094919    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:27:52.096515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.096487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" event={"ID":"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f","Type":"ContainerStarted","Data":"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"}
Jul 28 18:27:52.096613 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.096544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:27:52.096613 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.096577    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" event={"ID":"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f","Type":"ContainerStarted","Data":"4dc126d2e16c39b7530e330bc095da50093cd8a27158d8b1e02078fdd1c3c1a5"}
Jul 28 18:27:52.110648 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.110593    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" podStartSLOduration=2.722596472 podStartE2EDuration="3.110577221s" totalImagesPullingTime="387.980749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:52.109075218 +0000 UTC m=+1851.679138046" watchObservedRunningTime="2026-07-28 18:27:52.110577221 +0000 UTC m=+1851.680640243"
Jul 28 18:27:52.124545 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.124489    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" podStartSLOduration=2.12446909 podStartE2EDuration="2.12446909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:52.12383682 +0000 UTC m=+1851.693899663" watchObservedRunningTime="2026-07-28 18:27:52.12446909 +0000 UTC m=+1851.694531918"
Jul 28 18:27:52.141399 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:52.141345    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" podStartSLOduration=2.141329292 podStartE2EDuration="2.141329292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:27:52.139420312 +0000 UTC m=+1851.709483135" watchObservedRunningTime="2026-07-28 18:27:52.141329292 +0000 UTC m=+1851.711392123"
Jul 28 18:27:58.109159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:27:58.109128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:28:03.104459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:03.104421    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:28:03.104853 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:03.104510    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:28:20.366449 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.366413    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:28:20.366878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.366678    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" podUID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" containerName="maas-api" containerID="cri-o://332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e" gracePeriod=30
Jul 28 18:28:20.442156 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.442120    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:28:20.442409 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.442381    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" podUID="785949fd-0f51-47b1-b6bd-33d051041eb8" containerName="payload-processing" containerID="cri-o://b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d" gracePeriod=30
Jul 28 18:28:20.460418 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.460217    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:28:20.460534 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.460503    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" podUID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" containerName="payload-pre-processing" containerID="cri-o://5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9" gracePeriod=30
Jul 28 18:28:20.712109 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.712079    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:28:20.746412 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.746388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:28:20.750938 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.750917    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:28:20.786129 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.786091    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92btq\" (UniqueName: \"kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq\") pod \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") "
Jul 28 18:28:20.786318 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.786175    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") pod \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") "
Jul 28 18:28:20.786318 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.786228    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca\") pod \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\" (UID: \"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe\") "
Jul 28 18:28:20.786717 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.786686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" (UID: "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:20.788487 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.788452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" (UID: "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:28:20.788615 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.788493    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq" (OuterVolumeSpecName: "kube-api-access-92btq") pod "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" (UID: "8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe"). InnerVolumeSpecName "kube-api-access-92btq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:20.886848 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.886807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume\") pod \"785949fd-0f51-47b1-b6bd-33d051041eb8\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") "
Jul 28 18:28:20.887030 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.886918    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g77w9\" (UniqueName: \"kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9\") pod \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") "
Jul 28 18:28:20.887030 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.886942    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume\") pod \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\" (UID: \"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f\") "
Jul 28 18:28:20.887128 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8l95\" (UniqueName: \"kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95\") pod \"785949fd-0f51-47b1-b6bd-33d051041eb8\" (UID: \"785949fd-0f51-47b1-b6bd-33d051041eb8\") "
Jul 28 18:28:20.887315 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887297    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92btq\" (UniqueName: \"kubernetes.io/projected/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-kube-api-access-92btq\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.887420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887318    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.887420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887331    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.887420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "785949fd-0f51-47b1-b6bd-33d051041eb8" (UID: "785949fd-0f51-47b1-b6bd-33d051041eb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:20.887420 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.887370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" (UID: "3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:20.889207 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.889173    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95" (OuterVolumeSpecName: "kube-api-access-m8l95") pod "785949fd-0f51-47b1-b6bd-33d051041eb8" (UID: "785949fd-0f51-47b1-b6bd-33d051041eb8"). InnerVolumeSpecName "kube-api-access-m8l95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:20.889313 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.889199    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9" (OuterVolumeSpecName: "kube-api-access-g77w9") pod "3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" (UID: "3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f"). InnerVolumeSpecName "kube-api-access-g77w9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:20.988190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.988150    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8l95\" (UniqueName: \"kubernetes.io/projected/785949fd-0f51-47b1-b6bd-33d051041eb8-kube-api-access-m8l95\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.988190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.988181    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/785949fd-0f51-47b1-b6bd-33d051041eb8-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.988190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.988195    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g77w9\" (UniqueName: \"kubernetes.io/projected/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-kube-api-access-g77w9\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:20.988422 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:20.988207    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:21.236340 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.236246    2581 generic.go:356] "Generic (PLEG): container finished" podID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" containerID="332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e" exitCode=0
Jul 28 18:28:21.236340 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.236318    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"
Jul 28 18:28:21.236558 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.236336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" event={"ID":"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe","Type":"ContainerDied","Data":"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"}
Jul 28 18:28:21.236558 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.236387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz" event={"ID":"8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe","Type":"ContainerDied","Data":"681ead7213bac5ccb7a4ad1a0019d4281c941091646129b885aea15785643739"}
Jul 28 18:28:21.236558 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.236410    2581 scope.go:117] "RemoveContainer" containerID="332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"
Jul 28 18:28:21.237709 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.237685    2581 generic.go:356] "Generic (PLEG): container finished" podID="785949fd-0f51-47b1-b6bd-33d051041eb8" containerID="b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d" exitCode=0
Jul 28 18:28:21.237833 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.237723    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"
Jul 28 18:28:21.237833 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.237769    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" event={"ID":"785949fd-0f51-47b1-b6bd-33d051041eb8","Type":"ContainerDied","Data":"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"}
Jul 28 18:28:21.237833 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.237821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf" event={"ID":"785949fd-0f51-47b1-b6bd-33d051041eb8","Type":"ContainerDied","Data":"202c108934e4f628edaca9ea79c0b58ded4cabaa189ab62e94d86a57a652442d"}
Jul 28 18:28:21.239298 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.239257    2581 generic.go:356] "Generic (PLEG): container finished" podID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" containerID="5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9" exitCode=0
Jul 28 18:28:21.239405 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.239313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" event={"ID":"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f","Type":"ContainerDied","Data":"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"}
Jul 28 18:28:21.239405 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.239329    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"
Jul 28 18:28:21.239405 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.239369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb" event={"ID":"3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f","Type":"ContainerDied","Data":"4dc126d2e16c39b7530e330bc095da50093cd8a27158d8b1e02078fdd1c3c1a5"}
Jul 28 18:28:21.246177 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.246161    2581 scope.go:117] "RemoveContainer" containerID="332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"
Jul 28 18:28:21.246481 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:21.246463    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e\": container with ID starting with 332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e not found: ID does not exist" containerID="332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"
Jul 28 18:28:21.246542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.246490    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e"} err="failed to get container status \"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e\": rpc error: code = NotFound desc = could not find container \"332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e\": container with ID starting with 332a640678facb92213db93f28b33d6b02dff5773503d6a1687ebc1a9895459e not found: ID does not exist"
Jul 28 18:28:21.246542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.246506    2581 scope.go:117] "RemoveContainer" containerID="b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"
Jul 28 18:28:21.254180 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.254164    2581 scope.go:117] "RemoveContainer" containerID="b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"
Jul 28 18:28:21.254515 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:21.254493    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d\": container with ID starting with b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d not found: ID does not exist" containerID="b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"
Jul 28 18:28:21.254580 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.254521    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d"} err="failed to get container status \"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d\": rpc error: code = NotFound desc = could not find container \"b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d\": container with ID starting with b1fb1e222a1fadca56f0404581a287b459058248d071b058b427f3f98343495d not found: ID does not exist"
Jul 28 18:28:21.254580 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.254537    2581 scope.go:117] "RemoveContainer" containerID="5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"
Jul 28 18:28:21.261992 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.261969    2581 scope.go:117] "RemoveContainer" containerID="5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"
Jul 28 18:28:21.262244 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:21.262225    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9\": container with ID starting with 5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9 not found: ID does not exist" containerID="5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"
Jul 28 18:28:21.262331 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.262256    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9"} err="failed to get container status \"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9\": rpc error: code = NotFound desc = could not find container \"5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9\": container with ID starting with 5402fd7e8b2933e55e84d3f759c9d30f547eea4bd6bceb803c508e0a1c5dd0d9 not found: ID does not exist"
Jul 28 18:28:21.265628 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.265605    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:28:21.269551 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.269530    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3adc829c-b57c969cb-5gndz"]
Jul 28 18:28:21.280965 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.280938    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:28:21.285008 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.284986    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3adc829c-848c49c4fb-qqcnb"]
Jul 28 18:28:21.296457 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.296423    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:28:21.301496 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:21.301471    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-3adc829c-6699644f9c-q5snf"]
Jul 28 18:28:22.964775 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:22.964699    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" path="/var/lib/kubelet/pods/3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f/volumes"
Jul 28 18:28:22.965135 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:22.965064    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="785949fd-0f51-47b1-b6bd-33d051041eb8" path="/var/lib/kubelet/pods/785949fd-0f51-47b1-b6bd-33d051041eb8/volumes"
Jul 28 18:28:22.965459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:22.965444    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" path="/var/lib/kubelet/pods/8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe/volumes"
Jul 28 18:28:24.626346 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626302    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626656    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" containerName="maas-api"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626667    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" containerName="maas-api"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626678    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="785949fd-0f51-47b1-b6bd-33d051041eb8" containerName="payload-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626684    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="785949fd-0f51-47b1-b6bd-33d051041eb8" containerName="payload-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626690    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" containerName="payload-pre-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626696    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" containerName="payload-pre-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626751    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b0f2a82-83a1-4faf-9ceb-3bf6f7114d5f" containerName="payload-pre-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626761    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="785949fd-0f51-47b1-b6bd-33d051041eb8" containerName="payload-processing"
Jul 28 18:28:24.626818 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.626768    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8edfcc8e-2d5a-46d5-b8fc-356e6e6a62fe" containerName="maas-api"
Jul 28 18:28:24.631832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.631803    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.635243 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.635216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-223d27c2-serving-cert\""
Jul 28 18:28:24.645147 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.645085    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:24.724294 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.724230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.724475 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.724300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.724475 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.724395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6rp\" (UniqueName: \"kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.825406 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.825365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.825406 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.825408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.825655 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.825433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6rp\" (UniqueName: \"kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.825655 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:24.825549    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-serving-cert: secret "maas-api-e2e-ait-223d27c2-serving-cert" not found
Jul 28 18:28:24.825655 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:24.825633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls podName:3a8825c2-7897-4bb0-8cd5-5d9c2efa3128 nodeName:}" failed. No retries permitted until 2026-07-28 18:28:25.325608995 +0000 UTC m=+1884.895671803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls") pod "maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" (UID: "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128") : secret "maas-api-e2e-ait-223d27c2-serving-cert" not found
Jul 28 18:28:24.826050 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.826029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:24.843334 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:24.843298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6rp\" (UniqueName: \"kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:25.330478 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.330442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:25.332920 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.332893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") pod \"maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:25.492303 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.492249    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:25.495928 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.495909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.499157 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.499134    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-223d27c2-dockercfg-lm6x6\""
Jul 28 18:28:25.499295 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.499174    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-223d27c2\""
Jul 28 18:28:25.515246 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.515213    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:25.520517 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.520487    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:25.524827 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.524795    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.534253 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.534226    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:25.545001 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.544976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:25.634539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.634498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hwr\" (UniqueName: \"kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.634995 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.634574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwkp\" (UniqueName: \"kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.634995 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.634618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.634995 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.634683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.701857 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.701832    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:25.704024 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:28:25.703998    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8825c2_7897_4bb0_8cd5_5d9c2efa3128.slice/crio-76c9c671e49f9ba235c342149e23accd3485a1f2ba6d5d26684f8dfbc82f7694 WatchSource:0}: Error finding container 76c9c671e49f9ba235c342149e23accd3485a1f2ba6d5d26684f8dfbc82f7694: Status 404 returned error can't find the container with id 76c9c671e49f9ba235c342149e23accd3485a1f2ba6d5d26684f8dfbc82f7694
Jul 28 18:28:25.735121 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.735085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.735296 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.735256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hwr\" (UniqueName: \"kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.735508 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.735347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwkp\" (UniqueName: \"kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.735508 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.735378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.735945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.735926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.736078 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.736058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.744734 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.744706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwkp\" (UniqueName: \"kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp\") pod \"payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") " pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.744847 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.744744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hwr\" (UniqueName: \"kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr\") pod \"payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.807768 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.807739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:25.839196 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.839165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:25.987390 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:25.987235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:25.990011 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:28:25.989983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc81858f_9b1b_4747_bd23_02c70661244d.slice/crio-1f7ea7041f3f8ef02b88036f55272968a5fb1bd1929d5cda2203aed0e1ae7201 WatchSource:0}: Error finding container 1f7ea7041f3f8ef02b88036f55272968a5fb1bd1929d5cda2203aed0e1ae7201: Status 404 returned error can't find the container with id 1f7ea7041f3f8ef02b88036f55272968a5fb1bd1929d5cda2203aed0e1ae7201
Jul 28 18:28:26.015717 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.015626    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:26.020213 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:28:26.020186    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468e9311_6f05_49a6_8a1b_71e9e123caa4.slice/crio-b07f3434da2209a74b854f852d2173ae62fe17ae6c3c5744ecada058c1427f0b WatchSource:0}: Error finding container b07f3434da2209a74b854f852d2173ae62fe17ae6c3c5744ecada058c1427f0b: Status 404 returned error can't find the container with id b07f3434da2209a74b854f852d2173ae62fe17ae6c3c5744ecada058c1427f0b
Jul 28 18:28:26.267793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.267665    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" event={"ID":"468e9311-6f05-49a6-8a1b-71e9e123caa4","Type":"ContainerStarted","Data":"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"}
Jul 28 18:28:26.267793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.267732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" event={"ID":"468e9311-6f05-49a6-8a1b-71e9e123caa4","Type":"ContainerStarted","Data":"b07f3434da2209a74b854f852d2173ae62fe17ae6c3c5744ecada058c1427f0b"}
Jul 28 18:28:26.267793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.267762    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:26.269652 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.269624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" event={"ID":"cc81858f-9b1b-4747-bd23-02c70661244d","Type":"ContainerStarted","Data":"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"}
Jul 28 18:28:26.269837 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.269675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" event={"ID":"cc81858f-9b1b-4747-bd23-02c70661244d","Type":"ContainerStarted","Data":"1f7ea7041f3f8ef02b88036f55272968a5fb1bd1929d5cda2203aed0e1ae7201"}
Jul 28 18:28:26.269837 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.269734    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:26.271392 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.271366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" event={"ID":"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128","Type":"ContainerStarted","Data":"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"}
Jul 28 18:28:26.271518 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.271412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" event={"ID":"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128","Type":"ContainerStarted","Data":"76c9c671e49f9ba235c342149e23accd3485a1f2ba6d5d26684f8dfbc82f7694"}
Jul 28 18:28:26.271518 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.271449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:26.288700 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.288645    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" podStartSLOduration=1.2886260809999999 podStartE2EDuration="1.288626081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:26.288059484 +0000 UTC m=+1885.858122316" watchObservedRunningTime="2026-07-28 18:28:26.288626081 +0000 UTC m=+1885.858688912"
Jul 28 18:28:26.311366 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.311312    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" podStartSLOduration=1.311296087 podStartE2EDuration="1.311296087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:26.310853011 +0000 UTC m=+1885.880915842" watchObservedRunningTime="2026-07-28 18:28:26.311296087 +0000 UTC m=+1885.881358915"
Jul 28 18:28:26.336280 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:26.336221    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" podStartSLOduration=1.88937014 podStartE2EDuration="2.336206141s" totalImagesPullingTime="446.836001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:28:26.334756105 +0000 UTC m=+1885.904818936" watchObservedRunningTime="2026-07-28 18:28:26.336206141 +0000 UTC m=+1885.906268971"
Jul 28 18:28:32.283458 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:32.283423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:37.278935 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:37.278889    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:37.279414 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:37.279258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:55.372899 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.372842    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:55.373533 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.373105    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" podUID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" containerName="maas-api" containerID="cri-o://3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2" gracePeriod=30
Jul 28 18:28:55.435416 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.435378    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:55.435929 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.435868    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" podUID="cc81858f-9b1b-4747-bd23-02c70661244d" containerName="payload-processing" containerID="cri-o://88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89" gracePeriod=30
Jul 28 18:28:55.446291 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.445638    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:55.446291 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.445887    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" podUID="468e9311-6f05-49a6-8a1b-71e9e123caa4" containerName="payload-pre-processing" containerID="cri-o://4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9" gracePeriod=30
Jul 28 18:28:55.702411 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.702380    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:55.718300 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.718257    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:55.750027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.750000    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:55.819299 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume\") pod \"cc81858f-9b1b-4747-bd23-02c70661244d\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") "
Jul 28 18:28:55.819299 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume\") pod \"468e9311-6f05-49a6-8a1b-71e9e123caa4\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") "
Jul 28 18:28:55.819562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819339    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlwkp\" (UniqueName: \"kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp\") pod \"cc81858f-9b1b-4747-bd23-02c70661244d\" (UID: \"cc81858f-9b1b-4747-bd23-02c70661244d\") "
Jul 28 18:28:55.819562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca\") pod \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") "
Jul 28 18:28:55.819562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8hwr\" (UniqueName: \"kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr\") pod \"468e9311-6f05-49a6-8a1b-71e9e123caa4\" (UID: \"468e9311-6f05-49a6-8a1b-71e9e123caa4\") "
Jul 28 18:28:55.819562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") pod \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") "
Jul 28 18:28:55.819562 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819482    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn6rp\" (UniqueName: \"kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp\") pod \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\" (UID: \"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128\") "
Jul 28 18:28:55.819836 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819684    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc81858f-9b1b-4747-bd23-02c70661244d" (UID: "cc81858f-9b1b-4747-bd23-02c70661244d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:55.819836 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819748    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" (UID: "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:55.819948 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819872    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.819948 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819893    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc81858f-9b1b-4747-bd23-02c70661244d-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.820044 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.819969    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "468e9311-6f05-49a6-8a1b-71e9e123caa4" (UID: "468e9311-6f05-49a6-8a1b-71e9e123caa4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:28:55.821575 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.821546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp" (OuterVolumeSpecName: "kube-api-access-nlwkp") pod "cc81858f-9b1b-4747-bd23-02c70661244d" (UID: "cc81858f-9b1b-4747-bd23-02c70661244d"). InnerVolumeSpecName "kube-api-access-nlwkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:55.821677 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.821583    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" (UID: "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:28:55.821891 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.821872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr" (OuterVolumeSpecName: "kube-api-access-r8hwr") pod "468e9311-6f05-49a6-8a1b-71e9e123caa4" (UID: "468e9311-6f05-49a6-8a1b-71e9e123caa4"). InnerVolumeSpecName "kube-api-access-r8hwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:55.821891 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.821880    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp" (OuterVolumeSpecName: "kube-api-access-pn6rp") pod "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" (UID: "3a8825c2-7897-4bb0-8cd5-5d9c2efa3128"). InnerVolumeSpecName "kube-api-access-pn6rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:28:55.920548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.920446    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468e9311-6f05-49a6-8a1b-71e9e123caa4-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.920548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.920490    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlwkp\" (UniqueName: \"kubernetes.io/projected/cc81858f-9b1b-4747-bd23-02c70661244d-kube-api-access-nlwkp\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.920548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.920501    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8hwr\" (UniqueName: \"kubernetes.io/projected/468e9311-6f05-49a6-8a1b-71e9e123caa4-kube-api-access-r8hwr\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.920548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.920511    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:55.920548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:55.920520    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn6rp\" (UniqueName: \"kubernetes.io/projected/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128-kube-api-access-pn6rp\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:28:56.423717 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.423681    2581 generic.go:356] "Generic (PLEG): container finished" podID="468e9311-6f05-49a6-8a1b-71e9e123caa4" containerID="4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9" exitCode=0
Jul 28 18:28:56.424159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.423740    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"
Jul 28 18:28:56.424159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.423751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" event={"ID":"468e9311-6f05-49a6-8a1b-71e9e123caa4","Type":"ContainerDied","Data":"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"}
Jul 28 18:28:56.424159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.423806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5" event={"ID":"468e9311-6f05-49a6-8a1b-71e9e123caa4","Type":"ContainerDied","Data":"b07f3434da2209a74b854f852d2173ae62fe17ae6c3c5744ecada058c1427f0b"}
Jul 28 18:28:56.424159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.423828    2581 scope.go:117] "RemoveContainer" containerID="4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"
Jul 28 18:28:56.425108 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.425090    2581 generic.go:356] "Generic (PLEG): container finished" podID="cc81858f-9b1b-4747-bd23-02c70661244d" containerID="88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89" exitCode=0
Jul 28 18:28:56.425192 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.425153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" event={"ID":"cc81858f-9b1b-4747-bd23-02c70661244d","Type":"ContainerDied","Data":"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"}
Jul 28 18:28:56.425192 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.425166    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"
Jul 28 18:28:56.425192 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.425185    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc" event={"ID":"cc81858f-9b1b-4747-bd23-02c70661244d","Type":"ContainerDied","Data":"1f7ea7041f3f8ef02b88036f55272968a5fb1bd1929d5cda2203aed0e1ae7201"}
Jul 28 18:28:56.426578 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.426556    2581 generic.go:356] "Generic (PLEG): container finished" podID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" containerID="3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2" exitCode=0
Jul 28 18:28:56.426684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.426583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" event={"ID":"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128","Type":"ContainerDied","Data":"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"}
Jul 28 18:28:56.426684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.426617    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"
Jul 28 18:28:56.426684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.426638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm" event={"ID":"3a8825c2-7897-4bb0-8cd5-5d9c2efa3128","Type":"ContainerDied","Data":"76c9c671e49f9ba235c342149e23accd3485a1f2ba6d5d26684f8dfbc82f7694"}
Jul 28 18:28:56.433787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.433766    2581 scope.go:117] "RemoveContainer" containerID="4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"
Jul 28 18:28:56.434091 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:56.434070    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9\": container with ID starting with 4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9 not found: ID does not exist" containerID="4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"
Jul 28 18:28:56.434190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.434101    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9"} err="failed to get container status \"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9\": rpc error: code = NotFound desc = could not find container \"4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9\": container with ID starting with 4a2827426b00393216ae49b8cc565fda5be9345e1238252cce9c7baf2d18f7b9 not found: ID does not exist"
Jul 28 18:28:56.434190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.434124    2581 scope.go:117] "RemoveContainer" containerID="88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"
Jul 28 18:28:56.443113 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.443092    2581 scope.go:117] "RemoveContainer" containerID="88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"
Jul 28 18:28:56.443378 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:56.443361    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89\": container with ID starting with 88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89 not found: ID does not exist" containerID="88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"
Jul 28 18:28:56.443429 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.443384    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89"} err="failed to get container status \"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89\": rpc error: code = NotFound desc = could not find container \"88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89\": container with ID starting with 88881d006688ad81432149e453548cab1f040c09e54340017ad095b9e5711a89 not found: ID does not exist"
Jul 28 18:28:56.443429 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.443400    2581 scope.go:117] "RemoveContainer" containerID="3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"
Jul 28 18:28:56.450945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.450925    2581 scope.go:117] "RemoveContainer" containerID="3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"
Jul 28 18:28:56.451433 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:28:56.451405    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2\": container with ID starting with 3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2 not found: ID does not exist" containerID="3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"
Jul 28 18:28:56.451539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.451441    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2"} err="failed to get container status \"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2\": rpc error: code = NotFound desc = could not find container \"3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2\": container with ID starting with 3a2414d438c656d8dc8c66faa4d83eaf9a8cfeef9b1ec596c2e421c2b28246f2 not found: ID does not exist"
Jul 28 18:28:56.452434 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.452413    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:56.459014 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.458993    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-223d27c2-d4c94cd96-2f7m5"]
Jul 28 18:28:56.471790 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.471767    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:56.477163 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.477142    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-223d27c2-5f55ddbbb6-49jxc"]
Jul 28 18:28:56.489646 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.489624    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:56.494746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.494713    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-223d27c2-59d7f5d5-j4vpm"]
Jul 28 18:28:56.963787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.963753    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" path="/var/lib/kubelet/pods/3a8825c2-7897-4bb0-8cd5-5d9c2efa3128/volumes"
Jul 28 18:28:56.964134 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.964122    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468e9311-6f05-49a6-8a1b-71e9e123caa4" path="/var/lib/kubelet/pods/468e9311-6f05-49a6-8a1b-71e9e123caa4/volumes"
Jul 28 18:28:56.964482 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:56.964469    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc81858f-9b1b-4747-bd23-02c70661244d" path="/var/lib/kubelet/pods/cc81858f-9b1b-4747-bd23-02c70661244d/volumes"
Jul 28 18:28:59.905826 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.905792    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:28:59.906361 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906342    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468e9311-6f05-49a6-8a1b-71e9e123caa4" containerName="payload-pre-processing"
Jul 28 18:28:59.906455 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906364    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="468e9311-6f05-49a6-8a1b-71e9e123caa4" containerName="payload-pre-processing"
Jul 28 18:28:59.906455 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906380    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" containerName="maas-api"
Jul 28 18:28:59.906455 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906388    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" containerName="maas-api"
Jul 28 18:28:59.906455 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906403    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc81858f-9b1b-4747-bd23-02c70661244d" containerName="payload-processing"
Jul 28 18:28:59.906455 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906412    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc81858f-9b1b-4747-bd23-02c70661244d" containerName="payload-processing"
Jul 28 18:28:59.906696 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906492    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc81858f-9b1b-4747-bd23-02c70661244d" containerName="payload-processing"
Jul 28 18:28:59.906696 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906509    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="468e9311-6f05-49a6-8a1b-71e9e123caa4" containerName="payload-pre-processing"
Jul 28 18:28:59.906696 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.906519    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8825c2-7897-4bb0-8cd5-5d9c2efa3128" containerName="maas-api"
Jul 28 18:28:59.911307 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.911284    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:28:59.916618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.916598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-49ca23b2-serving-cert\""
Jul 28 18:28:59.928299 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:28:59.928244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:29:00.061880 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.061844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2bh9\" (UniqueName: \"kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.062045 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.061885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.062045 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.061977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.163225 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.163133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.163413 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.163245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2bh9\" (UniqueName: \"kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.163413 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.163305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.163937 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.163913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.165890 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.165868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.184644 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.184609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2bh9\" (UniqueName: \"kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9\") pod \"maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.221664 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.221627    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:00.366736 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.366705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:29:00.369812 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:29:00.369782    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d87a4b_558f_49ea_9163_8153f9328b60.slice/crio-d26a1418913ea0e831c46a410ece8fc69d8262af7b06679a07061665668c6780 WatchSource:0}: Error finding container d26a1418913ea0e831c46a410ece8fc69d8262af7b06679a07061665668c6780: Status 404 returned error can't find the container with id d26a1418913ea0e831c46a410ece8fc69d8262af7b06679a07061665668c6780
Jul 28 18:29:00.449230 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.449144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" event={"ID":"19d87a4b-558f-49ea-9163-8153f9328b60","Type":"ContainerStarted","Data":"d26a1418913ea0e831c46a410ece8fc69d8262af7b06679a07061665668c6780"}
Jul 28 18:29:00.709236 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.709152    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:00.713760 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.713712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.716447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.716428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-49ca23b2-dockercfg-zppjf\""
Jul 28 18:29:00.716559 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.716429    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-49ca23b2\""
Jul 28 18:29:00.727559 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.727528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:00.749477 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.749443    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:00.753380 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.753358    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.765901 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.765870    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:00.869733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.869690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsxz2\" (UniqueName: \"kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.869733 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.869737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.869981 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.869757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.869981 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.869783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vgr\" (UniqueName: \"kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.970747 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.970664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsxz2\" (UniqueName: \"kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.970747 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.970721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.971350 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.970848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.971350 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.970908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vgr\" (UniqueName: \"kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.973172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.973151    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-49ca23b2\""
Jul 28 18:29:00.979054 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.979022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vgr\" (UniqueName: \"kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.979243 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.979228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsxz2\" (UniqueName: \"kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:00.982480 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.982457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume\") pod \"payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") " pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:00.982577 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:00.982529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:01.027761 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.027734    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-49ca23b2-dockercfg-zppjf\""
Jul 28 18:29:01.035975 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.035950    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:01.064939 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.064909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:01.205069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.205039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:01.208629 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:29:01.208587    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b55c4ce_da21_4e45_a943_23ac8d8a9ddf.slice/crio-565b7425deac57233a55f30a514bbf069ad2d1461a7accd295663ca9a266563d WatchSource:0}: Error finding container 565b7425deac57233a55f30a514bbf069ad2d1461a7accd295663ca9a266563d: Status 404 returned error can't find the container with id 565b7425deac57233a55f30a514bbf069ad2d1461a7accd295663ca9a266563d
Jul 28 18:29:01.232930 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.231163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:01.249526 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:29:01.249494    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b02e07_a386_4203_a983_f91953f4a4f3.slice/crio-0972fcf5718a546dd6c294cc31504b1c93090bbb54e0d4f79e66f016bdf5bedd WatchSource:0}: Error finding container 0972fcf5718a546dd6c294cc31504b1c93090bbb54e0d4f79e66f016bdf5bedd: Status 404 returned error can't find the container with id 0972fcf5718a546dd6c294cc31504b1c93090bbb54e0d4f79e66f016bdf5bedd
Jul 28 18:29:01.455454 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.455416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" event={"ID":"e0b02e07-a386-4203-a983-f91953f4a4f3","Type":"ContainerStarted","Data":"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"}
Jul 28 18:29:01.455626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.455505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" event={"ID":"e0b02e07-a386-4203-a983-f91953f4a4f3","Type":"ContainerStarted","Data":"0972fcf5718a546dd6c294cc31504b1c93090bbb54e0d4f79e66f016bdf5bedd"}
Jul 28 18:29:01.455626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.455533    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:01.456941 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.456916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" event={"ID":"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf","Type":"ContainerStarted","Data":"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"}
Jul 28 18:29:01.457069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.456962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" event={"ID":"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf","Type":"ContainerStarted","Data":"565b7425deac57233a55f30a514bbf069ad2d1461a7accd295663ca9a266563d"}
Jul 28 18:29:01.457069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.457013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:01.458391 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.458358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" event={"ID":"19d87a4b-558f-49ea-9163-8153f9328b60","Type":"ContainerStarted","Data":"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"}
Jul 28 18:29:01.458503 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.458485    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:01.474139 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.474043    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" podStartSLOduration=1.4740267249999999 podStartE2EDuration="1.474026725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:29:01.471767341 +0000 UTC m=+1921.041830185" watchObservedRunningTime="2026-07-28 18:29:01.474026725 +0000 UTC m=+1921.044089534"
Jul 28 18:29:01.491770 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.491721    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" podStartSLOduration=2.111057109 podStartE2EDuration="2.491704787s" totalImagesPullingTime="380.647678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:29:01.48969094 +0000 UTC m=+1921.059753772" watchObservedRunningTime="2026-07-28 18:29:01.491704787 +0000 UTC m=+1921.061767617"
Jul 28 18:29:01.507437 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:01.507387    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" podStartSLOduration=1.507371264 podStartE2EDuration="1.507371264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:29:01.504834647 +0000 UTC m=+1921.074897477" watchObservedRunningTime="2026-07-28 18:29:01.507371264 +0000 UTC m=+1921.077434150"
Jul 28 18:29:07.471298 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:07.471246    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:12.466197 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:12.466163    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:12.466654 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:12.466410    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:30.656288 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.656231    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:29:30.656909 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.656542    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" podUID="19d87a4b-558f-49ea-9163-8153f9328b60" containerName="maas-api" containerID="cri-o://a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285" gracePeriod=30
Jul 28 18:29:30.710665 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.710634    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:30.710882 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.710850    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" podUID="e0b02e07-a386-4203-a983-f91953f4a4f3" containerName="payload-processing" containerID="cri-o://6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74" gracePeriod=30
Jul 28 18:29:30.717793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.717761    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:30.718044 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:30.717990    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" podUID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" containerName="payload-pre-processing" containerID="cri-o://4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1" gracePeriod=30
Jul 28 18:29:31.029001 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.028976    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:31.032543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.032521    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:31.036566 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.036537    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:31.132844 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.132812    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca\") pod \"19d87a4b-558f-49ea-9163-8153f9328b60\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") "
Jul 28 18:29:31.133015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.132878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls\") pod \"19d87a4b-558f-49ea-9163-8153f9328b60\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") "
Jul 28 18:29:31.133015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.132955    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume\") pod \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") "
Jul 28 18:29:31.133015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.132988    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume\") pod \"e0b02e07-a386-4203-a983-f91953f4a4f3\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") "
Jul 28 18:29:31.133188 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsxz2\" (UniqueName: \"kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2\") pod \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\" (UID: \"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf\") "
Jul 28 18:29:31.133188 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133075    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2bh9\" (UniqueName: \"kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9\") pod \"19d87a4b-558f-49ea-9163-8153f9328b60\" (UID: \"19d87a4b-558f-49ea-9163-8153f9328b60\") "
Jul 28 18:29:31.133188 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133101    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2vgr\" (UniqueName: \"kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr\") pod \"e0b02e07-a386-4203-a983-f91953f4a4f3\" (UID: \"e0b02e07-a386-4203-a983-f91953f4a4f3\") "
Jul 28 18:29:31.133384 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133188    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19d87a4b-558f-49ea-9163-8153f9328b60" (UID: "19d87a4b-558f-49ea-9163-8153f9328b60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:29:31.133447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" (UID: "5b55c4ce-da21-4e45-a943-23ac8d8a9ddf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:29:31.133502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133440    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19d87a4b-558f-49ea-9163-8153f9328b60-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.133502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.133470    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0b02e07-a386-4203-a983-f91953f4a4f3" (UID: "e0b02e07-a386-4203-a983-f91953f4a4f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:29:31.135367 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.135337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19d87a4b-558f-49ea-9163-8153f9328b60" (UID: "19d87a4b-558f-49ea-9163-8153f9328b60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:29:31.135479 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.135369    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9" (OuterVolumeSpecName: "kube-api-access-l2bh9") pod "19d87a4b-558f-49ea-9163-8153f9328b60" (UID: "19d87a4b-558f-49ea-9163-8153f9328b60"). InnerVolumeSpecName "kube-api-access-l2bh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:29:31.135674 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.135648    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr" (OuterVolumeSpecName: "kube-api-access-l2vgr") pod "e0b02e07-a386-4203-a983-f91953f4a4f3" (UID: "e0b02e07-a386-4203-a983-f91953f4a4f3"). InnerVolumeSpecName "kube-api-access-l2vgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:29:31.135789 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.135674    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2" (OuterVolumeSpecName: "kube-api-access-wsxz2") pod "5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" (UID: "5b55c4ce-da21-4e45-a943-23ac8d8a9ddf"). InnerVolumeSpecName "kube-api-access-wsxz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.233977    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19d87a4b-558f-49ea-9163-8153f9328b60-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.234021    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.234033    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0b02e07-a386-4203-a983-f91953f4a4f3-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.234042    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsxz2\" (UniqueName: \"kubernetes.io/projected/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf-kube-api-access-wsxz2\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.234052    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2bh9\" (UniqueName: \"kubernetes.io/projected/19d87a4b-558f-49ea-9163-8153f9328b60-kube-api-access-l2bh9\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.234075 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.234061    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2vgr\" (UniqueName: \"kubernetes.io/projected/e0b02e07-a386-4203-a983-f91953f4a4f3-kube-api-access-l2vgr\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:29:31.614227 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.614194    2581 generic.go:356] "Generic (PLEG): container finished" podID="19d87a4b-558f-49ea-9163-8153f9328b60" containerID="a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285" exitCode=0
Jul 28 18:29:31.614440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.614238    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"
Jul 28 18:29:31.614440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.614306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" event={"ID":"19d87a4b-558f-49ea-9163-8153f9328b60","Type":"ContainerDied","Data":"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"}
Jul 28 18:29:31.614440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.614370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8" event={"ID":"19d87a4b-558f-49ea-9163-8153f9328b60","Type":"ContainerDied","Data":"d26a1418913ea0e831c46a410ece8fc69d8262af7b06679a07061665668c6780"}
Jul 28 18:29:31.614440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.614404    2581 scope.go:117] "RemoveContainer" containerID="a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"
Jul 28 18:29:31.615628 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.615608    2581 generic.go:356] "Generic (PLEG): container finished" podID="e0b02e07-a386-4203-a983-f91953f4a4f3" containerID="6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74" exitCode=0
Jul 28 18:29:31.615691 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.615664    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"
Jul 28 18:29:31.615746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.615695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" event={"ID":"e0b02e07-a386-4203-a983-f91953f4a4f3","Type":"ContainerDied","Data":"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"}
Jul 28 18:29:31.615799 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.615746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd" event={"ID":"e0b02e07-a386-4203-a983-f91953f4a4f3","Type":"ContainerDied","Data":"0972fcf5718a546dd6c294cc31504b1c93090bbb54e0d4f79e66f016bdf5bedd"}
Jul 28 18:29:31.617041 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.617018    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" containerID="4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1" exitCode=0
Jul 28 18:29:31.617172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.617081    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"
Jul 28 18:29:31.617172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.617054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" event={"ID":"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf","Type":"ContainerDied","Data":"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"}
Jul 28 18:29:31.617172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.617146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr" event={"ID":"5b55c4ce-da21-4e45-a943-23ac8d8a9ddf","Type":"ContainerDied","Data":"565b7425deac57233a55f30a514bbf069ad2d1461a7accd295663ca9a266563d"}
Jul 28 18:29:31.625559 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.625446    2581 scope.go:117] "RemoveContainer" containerID="a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"
Jul 28 18:29:31.625800 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:29:31.625773    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285\": container with ID starting with a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285 not found: ID does not exist" containerID="a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"
Jul 28 18:29:31.625863 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.625813    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285"} err="failed to get container status \"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285\": rpc error: code = NotFound desc = could not find container \"a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285\": container with ID starting with a3f311eadd23feaa810bc1285192cc4b5cd7a783e00966ea19581b657dea3285 not found: ID does not exist"
Jul 28 18:29:31.625863 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.625838    2581 scope.go:117] "RemoveContainer" containerID="6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"
Jul 28 18:29:31.634606 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.634588    2581 scope.go:117] "RemoveContainer" containerID="6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"
Jul 28 18:29:31.634900 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:29:31.634874    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74\": container with ID starting with 6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74 not found: ID does not exist" containerID="6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"
Jul 28 18:29:31.634946 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.634912    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74"} err="failed to get container status \"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74\": rpc error: code = NotFound desc = could not find container \"6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74\": container with ID starting with 6f4e077f5aa8b7e0229a6584537b3dc35f1a38ed9b751f2bb0a28df682c69f74 not found: ID does not exist"
Jul 28 18:29:31.634946 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.634935    2581 scope.go:117] "RemoveContainer" containerID="4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"
Jul 28 18:29:31.643389 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.643346    2581 scope.go:117] "RemoveContainer" containerID="4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"
Jul 28 18:29:31.643663 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:29:31.643643    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1\": container with ID starting with 4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1 not found: ID does not exist" containerID="4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"
Jul 28 18:29:31.643749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.643669    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1"} err="failed to get container status \"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1\": rpc error: code = NotFound desc = could not find container \"4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1\": container with ID starting with 4cb21a8d015b0fe59db31feb03d97e3ebaa0234f3b6d68756915f7d063acf3e1 not found: ID does not exist"
Jul 28 18:29:31.646474 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.646453    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:29:31.648166 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.648143    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-49ca23b2-847f79c6d9-vsxc8"]
Jul 28 18:29:31.659081 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.659044    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:31.661803 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.661774    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-49ca23b2-65c979457b-l7jlr"]
Jul 28 18:29:31.675899 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.675873    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:31.678850 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:31.678826    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-49ca23b2-d94bb47bb-kzwxd"]
Jul 28 18:29:32.964146 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:32.964107    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d87a4b-558f-49ea-9163-8153f9328b60" path="/var/lib/kubelet/pods/19d87a4b-558f-49ea-9163-8153f9328b60/volumes"
Jul 28 18:29:32.964522 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:32.964503    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" path="/var/lib/kubelet/pods/5b55c4ce-da21-4e45-a943-23ac8d8a9ddf/volumes"
Jul 28 18:29:32.964827 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:29:32.964814    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0b02e07-a386-4203-a983-f91953f4a4f3" path="/var/lib/kubelet/pods/e0b02e07-a386-4203-a983-f91953f4a4f3/volumes"
Jul 28 18:31:19.630678 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.630642    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.630993    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" containerName="payload-pre-processing"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631004    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" containerName="payload-pre-processing"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631015    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0b02e07-a386-4203-a983-f91953f4a4f3" containerName="payload-processing"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631021    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0b02e07-a386-4203-a983-f91953f4a4f3" containerName="payload-processing"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631032    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d87a4b-558f-49ea-9163-8153f9328b60" containerName="maas-api"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631038    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d87a4b-558f-49ea-9163-8153f9328b60" containerName="maas-api"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631093    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d87a4b-558f-49ea-9163-8153f9328b60" containerName="maas-api"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631110    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0b02e07-a386-4203-a983-f91953f4a4f3" containerName="payload-processing"
Jul 28 18:31:19.631347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.631116    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b55c4ce-da21-4e45-a943-23ac8d8a9ddf" containerName="payload-pre-processing"
Jul 28 18:31:19.633968 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.633943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.636265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.636245    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0540baca-serving-cert\""
Jul 28 18:31:19.646469 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.646442    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:31:19.730362 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.730318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.730553 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.730454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vv2\" (UniqueName: \"kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.730553 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.730496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.831776 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.831739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vv2\" (UniqueName: \"kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.831776 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.831778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.832003 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.831830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.832488 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.832469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.834387 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.834367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.839914 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.839889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vv2\" (UniqueName: \"kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2\") pod \"maas-api-e2e-mt-0540baca-7c8887bb57-9q7br\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:19.945104 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:19.945008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:20.098464 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.098434    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:31:20.101095 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:31:20.101060    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ccf12c_97ac_4d34_8f0d_7102e03c68a4.slice/crio-f259bde14e2956119f3cdbb1bb563373e708863ef0aae55606a734de04d8a9f5 WatchSource:0}: Error finding container f259bde14e2956119f3cdbb1bb563373e708863ef0aae55606a734de04d8a9f5: Status 404 returned error can't find the container with id f259bde14e2956119f3cdbb1bb563373e708863ef0aae55606a734de04d8a9f5
Jul 28 18:31:20.115695 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.115659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" event={"ID":"12ccf12c-97ac-4d34-8f0d-7102e03c68a4","Type":"ContainerStarted","Data":"f259bde14e2956119f3cdbb1bb563373e708863ef0aae55606a734de04d8a9f5"}
Jul 28 18:31:20.673240 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.673188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:31:20.676527 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.676506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.679686 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.679663    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0540baca\""
Jul 28 18:31:20.679971 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.679954    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0540baca-dockercfg-mc8vc\""
Jul 28 18:31:20.689763 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.689735    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:31:20.742882 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.742832    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:31:20.746812 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.746791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.758803 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.758770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:31:20.841978 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.841946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9zzm\" (UniqueName: \"kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.842153 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.841992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.842153 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.842097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44xx4\" (UniqueName: \"kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.842153 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.842133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.943367 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.943328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44xx4\" (UniqueName: \"kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.943567 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.943376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.943567 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.943446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9zzm\" (UniqueName: \"kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.943567 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.943476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.944399 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.944375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.944458 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.944375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.958336 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.958302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44xx4\" (UniqueName: \"kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4\") pod \"payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:20.958539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.958521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9zzm\" (UniqueName: \"kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm\") pod \"payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:20.986441 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:20.986405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:21.058164 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.058122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:21.129711 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.128691    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" event={"ID":"12ccf12c-97ac-4d34-8f0d-7102e03c68a4","Type":"ContainerStarted","Data":"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"}
Jul 28 18:31:21.129711 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.129618    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:21.139651 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.139415    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:31:21.143342 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:31:21.143307    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a8fa97_04bc_44c8_b2a2_6f7305e4fcb3.slice/crio-8a3208cf061518f54543afa86a59903d72c85b9ded16f8e29483b5f7cdf7a376 WatchSource:0}: Error finding container 8a3208cf061518f54543afa86a59903d72c85b9ded16f8e29483b5f7cdf7a376: Status 404 returned error can't find the container with id 8a3208cf061518f54543afa86a59903d72c85b9ded16f8e29483b5f7cdf7a376
Jul 28 18:31:21.153327 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.153122    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" podStartSLOduration=1.732220137 podStartE2EDuration="2.153102868s" totalImagesPullingTime="420.882731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:31:21.150219809 +0000 UTC m=+2060.720282638" watchObservedRunningTime="2026-07-28 18:31:21.153102868 +0000 UTC m=+2060.723165701"
Jul 28 18:31:21.225662 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:21.225637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:31:21.229245 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:31:21.229219    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f78db8_13fb_4321_a234_5997fe768e0c.slice/crio-abecff65637f0882381103935cea3f6228fca84563889606b0dfc07ed7c975e5 WatchSource:0}: Error finding container abecff65637f0882381103935cea3f6228fca84563889606b0dfc07ed7c975e5: Status 404 returned error can't find the container with id abecff65637f0882381103935cea3f6228fca84563889606b0dfc07ed7c975e5
Jul 28 18:31:22.134534 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.134496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" event={"ID":"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3","Type":"ContainerStarted","Data":"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"}
Jul 28 18:31:22.135015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.134548    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" event={"ID":"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3","Type":"ContainerStarted","Data":"8a3208cf061518f54543afa86a59903d72c85b9ded16f8e29483b5f7cdf7a376"}
Jul 28 18:31:22.135015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.134670    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:31:22.136025 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.135991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" event={"ID":"e0f78db8-13fb-4321-a234-5997fe768e0c","Type":"ContainerStarted","Data":"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"}
Jul 28 18:31:22.136166 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.136040    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" event={"ID":"e0f78db8-13fb-4321-a234-5997fe768e0c","Type":"ContainerStarted","Data":"abecff65637f0882381103935cea3f6228fca84563889606b0dfc07ed7c975e5"}
Jul 28 18:31:22.136214 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.136185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:22.170471 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.170415    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" podStartSLOduration=2.17039592 podStartE2EDuration="2.17039592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:31:22.168950406 +0000 UTC m=+2061.739013235" watchObservedRunningTime="2026-07-28 18:31:22.17039592 +0000 UTC m=+2061.740458751"
Jul 28 18:31:22.196428 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:22.196372    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" podStartSLOduration=2.196355624 podStartE2EDuration="2.196355624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:31:22.192440146 +0000 UTC m=+2061.762502990" watchObservedRunningTime="2026-07-28 18:31:22.196355624 +0000 UTC m=+2061.766418487"
Jul 28 18:31:28.146549 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:28.146522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:31:33.142866 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:33.142825    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.45:9004: connect: connection refused"
Jul 28 18:31:33.143444 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:33.143421    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:31:43.141473 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:31:43.141442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:32:02.746502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.746462    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:32:02.749109 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.746727    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" podUID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" containerName="maas-api" containerID="cri-o://b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4" gracePeriod=30
Jul 28 18:32:02.883195 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.883159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:32:02.883719 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.883687    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerName="payload-processing" containerID="cri-o://b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994" gracePeriod=30
Jul 28 18:32:02.904072 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.903400    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:32:02.904072 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:02.903658    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing" containerID="cri-o://25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a" gracePeriod=30
Jul 28 18:32:03.036677 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.036650    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:32:03.132773 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.132736    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca\") pod \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") "
Jul 28 18:32:03.132947 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.132819    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9vv2\" (UniqueName: \"kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2\") pod \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") "
Jul 28 18:32:03.132947 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.132896    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls\") pod \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\" (UID: \"12ccf12c-97ac-4d34-8f0d-7102e03c68a4\") "
Jul 28 18:32:03.133164 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.133137    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "12ccf12c-97ac-4d34-8f0d-7102e03c68a4" (UID: "12ccf12c-97ac-4d34-8f0d-7102e03c68a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:03.133265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.133239    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.135505 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.135474    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2" (OuterVolumeSpecName: "kube-api-access-d9vv2") pod "12ccf12c-97ac-4d34-8f0d-7102e03c68a4" (UID: "12ccf12c-97ac-4d34-8f0d-7102e03c68a4"). InnerVolumeSpecName "kube-api-access-d9vv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:03.135592 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.135477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "12ccf12c-97ac-4d34-8f0d-7102e03c68a4" (UID: "12ccf12c-97ac-4d34-8f0d-7102e03c68a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:32:03.179823 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.179795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:32:03.185381 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.185358    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:32:03.234422 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.234392    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9vv2\" (UniqueName: \"kubernetes.io/projected/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-kube-api-access-d9vv2\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.234422 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.234419    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12ccf12c-97ac-4d34-8f0d-7102e03c68a4-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.327502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.327467    2581 generic.go:356] "Generic (PLEG): container finished" podID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerID="25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a" exitCode=0
Jul 28 18:32:03.327698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.327512    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"
Jul 28 18:32:03.327698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.327551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" event={"ID":"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3","Type":"ContainerDied","Data":"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"}
Jul 28 18:32:03.327698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.327608    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" event={"ID":"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3","Type":"ContainerDied","Data":"8a3208cf061518f54543afa86a59903d72c85b9ded16f8e29483b5f7cdf7a376"}
Jul 28 18:32:03.327698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.327640    2581 scope.go:117] "RemoveContainer" containerID="25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"
Jul 28 18:32:03.328828 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.328810    2581 generic.go:356] "Generic (PLEG): container finished" podID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerID="b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994" exitCode=0
Jul 28 18:32:03.328925 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.328848    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"
Jul 28 18:32:03.328925 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.328893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" event={"ID":"e0f78db8-13fb-4321-a234-5997fe768e0c","Type":"ContainerDied","Data":"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"}
Jul 28 18:32:03.329024 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.328942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" event={"ID":"e0f78db8-13fb-4321-a234-5997fe768e0c","Type":"ContainerDied","Data":"abecff65637f0882381103935cea3f6228fca84563889606b0dfc07ed7c975e5"}
Jul 28 18:32:03.330167 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.330142    2581 generic.go:356] "Generic (PLEG): container finished" podID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" containerID="b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4" exitCode=0
Jul 28 18:32:03.330248 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.330172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"
Jul 28 18:32:03.330248 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.330227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" event={"ID":"12ccf12c-97ac-4d34-8f0d-7102e03c68a4","Type":"ContainerDied","Data":"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"}
Jul 28 18:32:03.330367 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.330297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br" event={"ID":"12ccf12c-97ac-4d34-8f0d-7102e03c68a4","Type":"ContainerDied","Data":"f259bde14e2956119f3cdbb1bb563373e708863ef0aae55606a734de04d8a9f5"}
Jul 28 18:32:03.334617 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.334595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume\") pod \"e0f78db8-13fb-4321-a234-5997fe768e0c\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") "
Jul 28 18:32:03.334730 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.334645    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9zzm\" (UniqueName: \"kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm\") pod \"e0f78db8-13fb-4321-a234-5997fe768e0c\" (UID: \"e0f78db8-13fb-4321-a234-5997fe768e0c\") "
Jul 28 18:32:03.334730 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.334699    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44xx4\" (UniqueName: \"kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4\") pod \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") "
Jul 28 18:32:03.334864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.334779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume\") pod \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\" (UID: \"26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3\") "
Jul 28 18:32:03.335081 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.335052    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0f78db8-13fb-4321-a234-5997fe768e0c" (UID: "e0f78db8-13fb-4321-a234-5997fe768e0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:03.335345 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.335316    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" (UID: "26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:32:03.336923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.336901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm" (OuterVolumeSpecName: "kube-api-access-v9zzm") pod "e0f78db8-13fb-4321-a234-5997fe768e0c" (UID: "e0f78db8-13fb-4321-a234-5997fe768e0c"). InnerVolumeSpecName "kube-api-access-v9zzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:03.337256 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.337234    2581 scope.go:117] "RemoveContainer" containerID="25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"
Jul 28 18:32:03.337389 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.337366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4" (OuterVolumeSpecName: "kube-api-access-44xx4") pod "26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" (UID: "26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3"). InnerVolumeSpecName "kube-api-access-44xx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:32:03.337689 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:32:03.337667    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a\": container with ID starting with 25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a not found: ID does not exist" containerID="25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"
Jul 28 18:32:03.337757 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.337698    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a"} err="failed to get container status \"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a\": rpc error: code = NotFound desc = could not find container \"25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a\": container with ID starting with 25a6307078e00df75265bd6ed90be6d1f5b4fba1f2f22c3ada2659a01d8e885a not found: ID does not exist"
Jul 28 18:32:03.337757 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.337716    2581 scope.go:117] "RemoveContainer" containerID="b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"
Jul 28 18:32:03.345457 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.345439    2581 scope.go:117] "RemoveContainer" containerID="b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"
Jul 28 18:32:03.345713 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:32:03.345688    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994\": container with ID starting with b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994 not found: ID does not exist" containerID="b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"
Jul 28 18:32:03.345765 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.345722    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994"} err="failed to get container status \"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994\": rpc error: code = NotFound desc = could not find container \"b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994\": container with ID starting with b98e989c56a628f24de83d98d3882afbfa2b26ec142a10b8e977dd3a9141b994 not found: ID does not exist"
Jul 28 18:32:03.345765 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.345746    2581 scope.go:117] "RemoveContainer" containerID="b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"
Jul 28 18:32:03.351762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.351738    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:32:03.355306 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.355283    2581 scope.go:117] "RemoveContainer" containerID="b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"
Jul 28 18:32:03.355589 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:32:03.355571    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4\": container with ID starting with b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4 not found: ID does not exist" containerID="b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"
Jul 28 18:32:03.355656 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.355596    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4"} err="failed to get container status \"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4\": rpc error: code = NotFound desc = could not find container \"b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4\": container with ID starting with b7559d294cad3ad7633aefa9e1b0c270f3623733b0a6b69649a733f3d7a30ba4 not found: ID does not exist"
Jul 28 18:32:03.356098 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.356080    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0540baca-7c8887bb57-9q7br"]
Jul 28 18:32:03.435759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.435728    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9zzm\" (UniqueName: \"kubernetes.io/projected/e0f78db8-13fb-4321-a234-5997fe768e0c-kube-api-access-v9zzm\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.435759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.435752    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44xx4\" (UniqueName: \"kubernetes.io/projected/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-kube-api-access-44xx4\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.435759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.435763    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.435988 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.435772    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0f78db8-13fb-4321-a234-5997fe768e0c-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:32:03.653378 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.653344    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:32:03.659478 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.659450    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8"]
Jul 28 18:32:03.669642 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.669609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:32:03.677255 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:03.677228    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h"]
Jul 28 18:32:04.142466 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:04.142415    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-0540baca-864c4c74dd-jdk4h" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.45:9004: i/o timeout"
Jul 28 18:32:04.142935 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:04.142434    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-0540baca-775c77dcc8-g2ln8" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.46:9004: i/o timeout"
Jul 28 18:32:04.963765 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:04.963728    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" path="/var/lib/kubelet/pods/12ccf12c-97ac-4d34-8f0d-7102e03c68a4/volumes"
Jul 28 18:32:04.964105 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:04.964092    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" path="/var/lib/kubelet/pods/26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3/volumes"
Jul 28 18:32:04.964457 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:04.964444    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" path="/var/lib/kubelet/pods/e0f78db8-13fb-4321-a234-5997fe768e0c/volumes"
Jul 28 18:32:32.722801 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.722764    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:32:32.723407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723346    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" containerName="maas-api"
Jul 28 18:32:32.723407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723368    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" containerName="maas-api"
Jul 28 18:32:32.723407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723382    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing"
Jul 28 18:32:32.723407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723391    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing"
Jul 28 18:32:32.723627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723418    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerName="payload-processing"
Jul 28 18:32:32.723627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723428    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerName="payload-processing"
Jul 28 18:32:32.723627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a8fa97-04bc-44c8-b2a2-6f7305e4fcb3" containerName="payload-pre-processing"
Jul 28 18:32:32.723627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723556    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f78db8-13fb-4321-a234-5997fe768e0c" containerName="payload-processing"
Jul 28 18:32:32.723627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.723569    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ccf12c-97ac-4d34-8f0d-7102e03c68a4" containerName="maas-api"
Jul 28 18:32:32.726880 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.726856    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.729309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.729259    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-06daccab-serving-cert\""
Jul 28 18:32:32.737063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.737030    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:32:32.804821 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.804778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsf7h\" (UniqueName: \"kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.805059 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.804960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.805138 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.805114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.906610 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.906570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsf7h\" (UniqueName: \"kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.906885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.906626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.906885 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:32:32.906734    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-serving-cert: secret "maas-api-e2e-mt-06daccab-serving-cert" not found
Jul 28 18:32:32.906885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.906751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.906885 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:32:32.906790    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls podName:9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538 nodeName:}" failed. No retries permitted until 2026-07-28 18:32:33.406771608 +0000 UTC m=+2132.976834417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls") pod "maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" (UID: "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538") : secret "maas-api-e2e-mt-06daccab-serving-cert" not found
Jul 28 18:32:32.907379 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.907361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:32.918839 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:32.918805    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsf7h\" (UniqueName: \"kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:33.411754 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.411707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:33.412480 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.412458    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:32:33.414166 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.414146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") pod \"maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:33.415879 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.415859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.418698 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.418679    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-06daccab\""
Jul 28 18:32:33.427886 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.427855    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-06daccab-dockercfg-nz4xw\""
Jul 28 18:32:33.433265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.433242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:32:33.442581 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.442556    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:32:33.445772 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.445753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.482884 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.482708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:32:33.513295 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.513237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.513295 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.513290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6n4z\" (UniqueName: \"kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.513530 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.513365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.513530 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.513395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw446\" (UniqueName: \"kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.614539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.614497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.614539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.614535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw446\" (UniqueName: \"kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.614791 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.614685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.614791 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.614709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6n4z\" (UniqueName: \"kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.615425 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.615399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.615626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.615606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.624147 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.624117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw446\" (UniqueName: \"kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446\") pod \"payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") " pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.624354 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.624335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6n4z\" (UniqueName: \"kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z\") pod \"payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.639216 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.639184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:33.731697 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.731664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:33.760051 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.759986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:33.787031 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.786998    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:32:33.792485 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.792459    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:32:33.887851 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.887625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:32:33.891497 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:32:33.891469    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8a8a14_9816_4a61_b379_2fe4511da192.slice/crio-2b2941d17b18c490afd703724fe5b8870bd04af71c15cfcb94b8b1feae4b7b95 WatchSource:0}: Error finding container 2b2941d17b18c490afd703724fe5b8870bd04af71c15cfcb94b8b1feae4b7b95: Status 404 returned error can't find the container with id 2b2941d17b18c490afd703724fe5b8870bd04af71c15cfcb94b8b1feae4b7b95
Jul 28 18:32:33.921198 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:33.921165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:32:33.925065 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:32:33.925037    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13106a7_206a_41b7_8357_a075846168e9.slice/crio-2911bb284b653498713e9ef7a934747b2e79efc183a381ed65ecd4b858783a27 WatchSource:0}: Error finding container 2911bb284b653498713e9ef7a934747b2e79efc183a381ed65ecd4b858783a27: Status 404 returned error can't find the container with id 2911bb284b653498713e9ef7a934747b2e79efc183a381ed65ecd4b858783a27
Jul 28 18:32:34.481721 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.481605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" event={"ID":"a13106a7-206a-41b7-8357-a075846168e9","Type":"ContainerStarted","Data":"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"}
Jul 28 18:32:34.481721 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.481672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" event={"ID":"a13106a7-206a-41b7-8357-a075846168e9","Type":"ContainerStarted","Data":"2911bb284b653498713e9ef7a934747b2e79efc183a381ed65ecd4b858783a27"}
Jul 28 18:32:34.481951 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.481794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:34.483439 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.483411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" event={"ID":"ea8a8a14-9816-4a61-b379-2fe4511da192","Type":"ContainerStarted","Data":"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"}
Jul 28 18:32:34.483568 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.483465    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:32:34.483568 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.483486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" event={"ID":"ea8a8a14-9816-4a61-b379-2fe4511da192","Type":"ContainerStarted","Data":"2b2941d17b18c490afd703724fe5b8870bd04af71c15cfcb94b8b1feae4b7b95"}
Jul 28 18:32:34.485130 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.485106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" event={"ID":"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538","Type":"ContainerStarted","Data":"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"}
Jul 28 18:32:34.485226 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.485153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" event={"ID":"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538","Type":"ContainerStarted","Data":"1d21ac869fd061cfd55249a1cf68f3a145137805f563e1eb0e248d67fe74b921"}
Jul 28 18:32:34.485226 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.485181    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:34.499498 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.499450    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" podStartSLOduration=1.499435719 podStartE2EDuration="1.499435719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:32:34.497947075 +0000 UTC m=+2134.068009918" watchObservedRunningTime="2026-07-28 18:32:34.499435719 +0000 UTC m=+2134.069498550"
Jul 28 18:32:34.513957 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.513904    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" podStartSLOduration=1.513888122 podStartE2EDuration="1.513888122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:32:34.513212569 +0000 UTC m=+2134.083275404" watchObservedRunningTime="2026-07-28 18:32:34.513888122 +0000 UTC m=+2134.083950953"
Jul 28 18:32:34.531676 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:34.531613    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" podStartSLOduration=2.092790618 podStartE2EDuration="2.531593298s" totalImagesPullingTime="438.80268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:32:34.52985351 +0000 UTC m=+2134.099916340" watchObservedRunningTime="2026-07-28 18:32:34.531593298 +0000 UTC m=+2134.101656129"
Jul 28 18:32:40.495036 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:40.494999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:32:45.492430 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:45.492402    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:32:45.492839 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:32:45.492478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:33:52.949554 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:52.949520    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:33:52.950131 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:52.949783    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" podUID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" containerName="maas-api" containerID="cri-o://41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147" gracePeriod=30
Jul 28 18:33:53.005180 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.005148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:33:53.009770 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.009737    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:33:53.010047 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.010015    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" podUID="ea8a8a14-9816-4a61-b379-2fe4511da192" containerName="payload-pre-processing" containerID="cri-o://1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b" gracePeriod=30
Jul 28 18:33:53.089454 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.089420    2581 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-mt-06daccab: configmap "payload-processing-plugins-e2e-mt-06daccab" not found
Jul 28 18:33:53.089665 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.089528    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume podName:a13106a7-206a-41b7-8357-a075846168e9 nodeName:}" failed. No retries permitted until 2026-07-28 18:33:53.589507009 +0000 UTC m=+2213.159569818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume") pod "payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" (UID: "a13106a7-206a-41b7-8357-a075846168e9") : configmap "payload-processing-plugins-e2e-mt-06daccab" not found
Jul 28 18:33:53.271760 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.271735    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:33:53.274670 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.274651    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:33:53.392852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.392817    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume\") pod \"ea8a8a14-9816-4a61-b379-2fe4511da192\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") "
Jul 28 18:33:53.393057 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.392871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") pod \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") "
Jul 28 18:33:53.393057 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.392891    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca\") pod \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") "
Jul 28 18:33:53.393178 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsf7h\" (UniqueName: \"kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h\") pod \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\" (UID: \"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538\") "
Jul 28 18:33:53.393178 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6n4z\" (UniqueName: \"kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z\") pod \"ea8a8a14-9816-4a61-b379-2fe4511da192\" (UID: \"ea8a8a14-9816-4a61-b379-2fe4511da192\") "
Jul 28 18:33:53.393310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea8a8a14-9816-4a61-b379-2fe4511da192" (UID: "ea8a8a14-9816-4a61-b379-2fe4511da192"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:33:53.393310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393214    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" (UID: "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:33:53.393529 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393510    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a8a14-9816-4a61-b379-2fe4511da192-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:53.393588 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.393530    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:53.394999 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.394977    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" (UID: "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:33:53.395173 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.395156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h" (OuterVolumeSpecName: "kube-api-access-wsf7h") pod "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" (UID: "9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538"). InnerVolumeSpecName "kube-api-access-wsf7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:53.395284 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.395202    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z" (OuterVolumeSpecName: "kube-api-access-r6n4z") pod "ea8a8a14-9816-4a61-b379-2fe4511da192" (UID: "ea8a8a14-9816-4a61-b379-2fe4511da192"). InnerVolumeSpecName "kube-api-access-r6n4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:53.494618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.494577    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:53.494618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.494613    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsf7h\" (UniqueName: \"kubernetes.io/projected/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538-kube-api-access-wsf7h\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:53.494618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.494628    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6n4z\" (UniqueName: \"kubernetes.io/projected/ea8a8a14-9816-4a61-b379-2fe4511da192-kube-api-access-r6n4z\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:53.596110 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.596068    2581 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-mt-06daccab: configmap "payload-processing-plugins-e2e-mt-06daccab" not found
Jul 28 18:33:53.596365 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.596161    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume podName:a13106a7-206a-41b7-8357-a075846168e9 nodeName:}" failed. No retries permitted until 2026-07-28 18:33:54.59614261 +0000 UTC m=+2214.166205420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume") pod "payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" (UID: "a13106a7-206a-41b7-8357-a075846168e9") : configmap "payload-processing-plugins-e2e-mt-06daccab" not found
Jul 28 18:33:53.859548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.859454    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea8a8a14-9816-4a61-b379-2fe4511da192" containerID="1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b" exitCode=0
Jul 28 18:33:53.859548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.859506    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"
Jul 28 18:33:53.859781 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.859541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" event={"ID":"ea8a8a14-9816-4a61-b379-2fe4511da192","Type":"ContainerDied","Data":"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"}
Jul 28 18:33:53.859781 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.859593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj" event={"ID":"ea8a8a14-9816-4a61-b379-2fe4511da192","Type":"ContainerDied","Data":"2b2941d17b18c490afd703724fe5b8870bd04af71c15cfcb94b8b1feae4b7b95"}
Jul 28 18:33:53.859781 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.859620    2581 scope.go:117] "RemoveContainer" containerID="1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"
Jul 28 18:33:53.860930 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.860904    2581 generic.go:356] "Generic (PLEG): container finished" podID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" containerID="41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147" exitCode=0
Jul 28 18:33:53.860930 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.860943    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"
Jul 28 18:33:53.861123 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.860973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" event={"ID":"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538","Type":"ContainerDied","Data":"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"}
Jul 28 18:33:53.861123 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.861015    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv" event={"ID":"9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538","Type":"ContainerDied","Data":"1d21ac869fd061cfd55249a1cf68f3a145137805f563e1eb0e248d67fe74b921"}
Jul 28 18:33:53.861251 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.861211    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" podUID="a13106a7-206a-41b7-8357-a075846168e9" containerName="payload-processing" containerID="cri-o://0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa" gracePeriod=30
Jul 28 18:33:53.870841 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.870812    2581 scope.go:117] "RemoveContainer" containerID="1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"
Jul 28 18:33:53.871139 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.871114    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b\": container with ID starting with 1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b not found: ID does not exist" containerID="1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"
Jul 28 18:33:53.871223 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.871145    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b"} err="failed to get container status \"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b\": rpc error: code = NotFound desc = could not find container \"1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b\": container with ID starting with 1f4d9a35e9f5329f558a59a5eb95274b044572e26dfc12d728f2b958e2d7ef5b not found: ID does not exist"
Jul 28 18:33:53.871223 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.871161    2581 scope.go:117] "RemoveContainer" containerID="41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"
Jul 28 18:33:53.896251 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.896218    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:33:53.902620 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.902592    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-06daccab-56bb8f4fb8-vknmj"]
Jul 28 18:33:53.914333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.914301    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:33:53.915844 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.915825    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06daccab-77bb54cdf5-7d4zv"]
Jul 28 18:33:53.972359 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.972242    2581 scope.go:117] "RemoveContainer" containerID="41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"
Jul 28 18:33:53.972693 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:53.972671    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147\": container with ID starting with 41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147 not found: ID does not exist" containerID="41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"
Jul 28 18:33:53.972742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:53.972703    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147"} err="failed to get container status \"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147\": rpc error: code = NotFound desc = could not find container \"41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147\": container with ID starting with 41c3d714465404211856a4dd799d1f4b8abcd2d26cff40f790dc95c73e969147 not found: ID does not exist"
Jul 28 18:33:54.119382 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.119317    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:33:54.302747 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.302711    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw446\" (UniqueName: \"kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446\") pod \"a13106a7-206a-41b7-8357-a075846168e9\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") "
Jul 28 18:33:54.302925 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.302821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume\") pod \"a13106a7-206a-41b7-8357-a075846168e9\" (UID: \"a13106a7-206a-41b7-8357-a075846168e9\") "
Jul 28 18:33:54.303213 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.303190    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a13106a7-206a-41b7-8357-a075846168e9" (UID: "a13106a7-206a-41b7-8357-a075846168e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:33:54.304769 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.304750    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446" (OuterVolumeSpecName: "kube-api-access-zw446") pod "a13106a7-206a-41b7-8357-a075846168e9" (UID: "a13106a7-206a-41b7-8357-a075846168e9"). InnerVolumeSpecName "kube-api-access-zw446". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:33:54.403775 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.403689    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a13106a7-206a-41b7-8357-a075846168e9-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:54.403775 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.403718    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw446\" (UniqueName: \"kubernetes.io/projected/a13106a7-206a-41b7-8357-a075846168e9-kube-api-access-zw446\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:33:54.867868 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.867829    2581 generic.go:356] "Generic (PLEG): container finished" podID="a13106a7-206a-41b7-8357-a075846168e9" containerID="0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa" exitCode=0
Jul 28 18:33:54.868027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.867880    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"
Jul 28 18:33:54.868027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.867891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" event={"ID":"a13106a7-206a-41b7-8357-a075846168e9","Type":"ContainerDied","Data":"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"}
Jul 28 18:33:54.868027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.867943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49" event={"ID":"a13106a7-206a-41b7-8357-a075846168e9","Type":"ContainerDied","Data":"2911bb284b653498713e9ef7a934747b2e79efc183a381ed65ecd4b858783a27"}
Jul 28 18:33:54.868027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.867973    2581 scope.go:117] "RemoveContainer" containerID="0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"
Jul 28 18:33:54.876810 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.876792    2581 scope.go:117] "RemoveContainer" containerID="0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"
Jul 28 18:33:54.877098 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:33:54.877078    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa\": container with ID starting with 0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa not found: ID does not exist" containerID="0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"
Jul 28 18:33:54.877147 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.877107    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa"} err="failed to get container status \"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa\": rpc error: code = NotFound desc = could not find container \"0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa\": container with ID starting with 0da3bcaac76bfd699bff0d6b125b1849463b26177fb73c6c122396f5f09e92fa not found: ID does not exist"
Jul 28 18:33:54.889777 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.889752    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:33:54.894972 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.894947    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-06daccab-5d85f7dd98-5bw49"]
Jul 28 18:33:54.963926 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.963894    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" path="/var/lib/kubelet/pods/9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538/volumes"
Jul 28 18:33:54.964281 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.964256    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a13106a7-206a-41b7-8357-a075846168e9" path="/var/lib/kubelet/pods/a13106a7-206a-41b7-8357-a075846168e9/volumes"
Jul 28 18:33:54.964606 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:33:54.964594    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8a8a14-9816-4a61-b379-2fe4511da192" path="/var/lib/kubelet/pods/ea8a8a14-9816-4a61-b379-2fe4511da192/volumes"
Jul 28 18:35:44.830069 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830029    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:35:44.830626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830585    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8a8a14-9816-4a61-b379-2fe4511da192" containerName="payload-pre-processing"
Jul 28 18:35:44.830626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830607    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8a8a14-9816-4a61-b379-2fe4511da192" containerName="payload-pre-processing"
Jul 28 18:35:44.830746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830629    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" containerName="maas-api"
Jul 28 18:35:44.830746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830638    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" containerName="maas-api"
Jul 28 18:35:44.830746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830668    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a13106a7-206a-41b7-8357-a075846168e9" containerName="payload-processing"
Jul 28 18:35:44.830746 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830677    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a13106a7-206a-41b7-8357-a075846168e9" containerName="payload-processing"
Jul 28 18:35:44.830941 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830765    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dfe8bfa-0eb6-4f60-a1eb-2c59f5b0b538" containerName="maas-api"
Jul 28 18:35:44.830941 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830780    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a13106a7-206a-41b7-8357-a075846168e9" containerName="payload-processing"
Jul 28 18:35:44.830941 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.830790    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8a8a14-9816-4a61-b379-2fe4511da192" containerName="payload-pre-processing"
Jul 28 18:35:44.833915 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.833892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.836013 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.835984    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b228ad-serving-cert\""
Jul 28 18:35:44.845005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.844977    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:35:44.897998 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.897929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.898211 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.898057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptcx\" (UniqueName: \"kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.898211 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.898087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.998926 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.998891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.999145 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.998944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptcx\" (UniqueName: \"kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.999145 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.999069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:44.999691 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:44.999662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:45.001619 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.001599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:45.019085 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.019049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptcx\" (UniqueName: \"kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx\") pod \"maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:45.145319 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.145213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:45.286312 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.286234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:35:45.289414 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:35:45.289385    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdeda853_0c7e_4201_ab8f_9704e4e102eb.slice/crio-a0440bf4ad0c687c400c696cba7bc3d0d95f9b0c0ef866dcf37e3986c2c3419b WatchSource:0}: Error finding container a0440bf4ad0c687c400c696cba7bc3d0d95f9b0c0ef866dcf37e3986c2c3419b: Status 404 returned error can't find the container with id a0440bf4ad0c687c400c696cba7bc3d0d95f9b0c0ef866dcf37e3986c2c3419b
Jul 28 18:35:45.379989 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.379950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" event={"ID":"fdeda853-0c7e-4201-ab8f-9704e4e102eb","Type":"ContainerStarted","Data":"a0440bf4ad0c687c400c696cba7bc3d0d95f9b0c0ef866dcf37e3986c2c3419b"}
Jul 28 18:35:45.580382 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.580345    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:35:45.584864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.584844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.589561 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.589535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b228ad\""
Jul 28 18:35:45.589686 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.589641    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b228ad-dockercfg-8n9tg\""
Jul 28 18:35:45.597809 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.597783    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:35:45.603193 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.603163    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:35:45.606559 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.606537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.620745 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.620715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:35:45.705127 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.705092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.705333 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.705226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q9ld\" (UniqueName: \"kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.705502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.705332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.705502 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.705400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrp8t\" (UniqueName: \"kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.806104 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.806309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrp8t\" (UniqueName: \"kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.806309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.806309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q9ld\" (UniqueName: \"kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.806912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806893    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.806960 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.806939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.814138 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.814114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q9ld\" (UniqueName: \"kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld\") pod \"payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.814254 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.814137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrp8t\" (UniqueName: \"kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t\") pod \"payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:45.894730 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.894703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:45.917810 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:45.917773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:46.060864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.060697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:35:46.064625 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:35:46.064427    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7766dc40_c248_4bde_ba16_8bb93adb2717.slice/crio-e86d94cec0a929b55a1f056baea85fb3a84b3828cceafbb739fa10b89de1b578 WatchSource:0}: Error finding container e86d94cec0a929b55a1f056baea85fb3a84b3828cceafbb739fa10b89de1b578: Status 404 returned error can't find the container with id e86d94cec0a929b55a1f056baea85fb3a84b3828cceafbb739fa10b89de1b578
Jul 28 18:35:46.297566 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.297327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:35:46.299942 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:35:46.299912    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4900c5_ba80_463e_b105_3eb92db4fe86.slice/crio-12a740143084c61d8710697d5e7d8af87af8975e6d6f4ae433ce2d0da5b33f52 WatchSource:0}: Error finding container 12a740143084c61d8710697d5e7d8af87af8975e6d6f4ae433ce2d0da5b33f52: Status 404 returned error can't find the container with id 12a740143084c61d8710697d5e7d8af87af8975e6d6f4ae433ce2d0da5b33f52
Jul 28 18:35:46.387006 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.386965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" event={"ID":"fdeda853-0c7e-4201-ab8f-9704e4e102eb","Type":"ContainerStarted","Data":"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"}
Jul 28 18:35:46.387309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.387289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:46.388871 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.388835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" event={"ID":"7766dc40-c248-4bde-ba16-8bb93adb2717","Type":"ContainerStarted","Data":"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"}
Jul 28 18:35:46.389021 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.388881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" event={"ID":"7766dc40-c248-4bde-ba16-8bb93adb2717","Type":"ContainerStarted","Data":"e86d94cec0a929b55a1f056baea85fb3a84b3828cceafbb739fa10b89de1b578"}
Jul 28 18:35:46.389021 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.388905    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:46.390459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.390433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" event={"ID":"6b4900c5-ba80-463e-b105-3eb92db4fe86","Type":"ContainerStarted","Data":"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"}
Jul 28 18:35:46.390570 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.390484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" event={"ID":"6b4900c5-ba80-463e-b105-3eb92db4fe86","Type":"ContainerStarted","Data":"12a740143084c61d8710697d5e7d8af87af8975e6d6f4ae433ce2d0da5b33f52"}
Jul 28 18:35:46.390570 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.390552    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:35:46.405772 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.405708    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" podStartSLOduration=1.870628948 podStartE2EDuration="2.405692202s" totalImagesPullingTime="535.063254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:35:46.403890688 +0000 UTC m=+2325.973953535" watchObservedRunningTime="2026-07-28 18:35:46.405692202 +0000 UTC m=+2325.975755034"
Jul 28 18:35:46.420175 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.420112    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" podStartSLOduration=1.420094965 podStartE2EDuration="1.420094965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:35:46.418721713 +0000 UTC m=+2325.988784545" watchObservedRunningTime="2026-07-28 18:35:46.420094965 +0000 UTC m=+2325.990157796"
Jul 28 18:35:46.434986 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:46.434937    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" podStartSLOduration=1.4349213490000001 podStartE2EDuration="1.434921349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:35:46.433308905 +0000 UTC m=+2326.003371735" watchObservedRunningTime="2026-07-28 18:35:46.434921349 +0000 UTC m=+2326.004984180"
Jul 28 18:35:52.400171 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:52.400142    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:35:57.397135 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:57.397047    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:35:57.397135 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:35:57.397134    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:36:18.975520 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:18.975489    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:36:18.979063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:18.979041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:18.982197 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:18.982175    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-210335-serving-cert\""
Jul 28 18:36:18.989015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:18.988962    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:36:19.008546 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.008503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.008737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.008555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdcm4\" (UniqueName: \"kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.008737 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.008634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.109478 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.109445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdcm4\" (UniqueName: \"kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.109710 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.109540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.109710 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.109634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.109710 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:36:19.109663    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-serving-cert: secret "maas-api-e2e-inf-b-210335-serving-cert" not found
Jul 28 18:36:19.109875 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:36:19.109740    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls podName:70534586-b6cd-4ab3-8d79-e885a6463265 nodeName:}" failed. No retries permitted until 2026-07-28 18:36:19.609719196 +0000 UTC m=+2359.179782005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls") pod "maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" (UID: "70534586-b6cd-4ab3-8d79-e885a6463265") : secret "maas-api-e2e-inf-b-210335-serving-cert" not found
Jul 28 18:36:19.110211 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.110194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.118915 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.118888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdcm4\" (UniqueName: \"kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.613672 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.613632    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.616431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.616405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") pod \"maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.702074 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.702037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:36:19.705726 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.705707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.708030 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.708008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-210335-dockercfg-k49qt\""
Jul 28 18:36:19.709532 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.709515    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-210335\""
Jul 28 18:36:19.718518 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.718493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:36:19.723839 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.723808    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:36:19.727771 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.727751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.741019 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.740997    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:36:19.815939 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.815895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.816107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.815949    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshw6\" (UniqueName: \"kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.816107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.816052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.816107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.816075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkhg\" (UniqueName: \"kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.891079 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.890998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:19.917465 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.917435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.917647 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.917475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshw6\" (UniqueName: \"kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.917647 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.917529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.917772 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.917646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkhg\" (UniqueName: \"kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.918316 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.918292    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.918524 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.918501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:19.932199 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.932171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkhg\" (UniqueName: \"kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg\") pod \"payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:19.932391 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:19.932173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshw6\" (UniqueName: \"kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6\") pod \"payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:20.017130 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.017100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:20.034923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.034896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:36:20.037893 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.037861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:20.038321 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:36:20.038290    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70534586_b6cd_4ab3_8d79_e885a6463265.slice/crio-fe69092b7c529b8a011aec960a26990bf7e60b9e5948a1ea9d2fdd3685db8a75 WatchSource:0}: Error finding container fe69092b7c529b8a011aec960a26990bf7e60b9e5948a1ea9d2fdd3685db8a75: Status 404 returned error can't find the container with id fe69092b7c529b8a011aec960a26990bf7e60b9e5948a1ea9d2fdd3685db8a75
Jul 28 18:36:20.186543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.183409    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:36:20.189448 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:36:20.189413    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aad389a_e02f_4887_b05d_7b5d3cb9d514.slice/crio-068c98cc2f0c630d1b66c17f0fec8747be5e35f083a6c2a7f916a461cc45bc07 WatchSource:0}: Error finding container 068c98cc2f0c630d1b66c17f0fec8747be5e35f083a6c2a7f916a461cc45bc07: Status 404 returned error can't find the container with id 068c98cc2f0c630d1b66c17f0fec8747be5e35f083a6c2a7f916a461cc45bc07
Jul 28 18:36:20.203407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.203382    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:36:20.205831 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:36:20.205803    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067dc2a6_14eb_4b77_8768_b480968b069e.slice/crio-4ec313e53e5094097e3218c8d889ca05fb320d86481ebdeeddcd92eed5de2177 WatchSource:0}: Error finding container 4ec313e53e5094097e3218c8d889ca05fb320d86481ebdeeddcd92eed5de2177: Status 404 returned error can't find the container with id 4ec313e53e5094097e3218c8d889ca05fb320d86481ebdeeddcd92eed5de2177
Jul 28 18:36:20.552082 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.551985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" event={"ID":"7aad389a-e02f-4887-b05d-7b5d3cb9d514","Type":"ContainerStarted","Data":"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"}
Jul 28 18:36:20.552082 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.552066    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:36:20.552311 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.552092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" event={"ID":"7aad389a-e02f-4887-b05d-7b5d3cb9d514","Type":"ContainerStarted","Data":"068c98cc2f0c630d1b66c17f0fec8747be5e35f083a6c2a7f916a461cc45bc07"}
Jul 28 18:36:20.553435 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.553409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" event={"ID":"067dc2a6-14eb-4b77-8768-b480968b069e","Type":"ContainerStarted","Data":"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"}
Jul 28 18:36:20.553539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.553464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:20.553539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.553487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" event={"ID":"067dc2a6-14eb-4b77-8768-b480968b069e","Type":"ContainerStarted","Data":"4ec313e53e5094097e3218c8d889ca05fb320d86481ebdeeddcd92eed5de2177"}
Jul 28 18:36:20.554659 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.554635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" event={"ID":"70534586-b6cd-4ab3-8d79-e885a6463265","Type":"ContainerStarted","Data":"fe69092b7c529b8a011aec960a26990bf7e60b9e5948a1ea9d2fdd3685db8a75"}
Jul 28 18:36:20.570726 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.570679    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" podStartSLOduration=1.570658865 podStartE2EDuration="1.570658865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:36:20.569195875 +0000 UTC m=+2360.139258709" watchObservedRunningTime="2026-07-28 18:36:20.570658865 +0000 UTC m=+2360.140721696"
Jul 28 18:36:20.591190 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:20.591145    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" podStartSLOduration=1.591129786 podStartE2EDuration="1.591129786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:36:20.587265418 +0000 UTC m=+2360.157328248" watchObservedRunningTime="2026-07-28 18:36:20.591129786 +0000 UTC m=+2360.161192617"
Jul 28 18:36:21.560492 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:21.560458    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" event={"ID":"70534586-b6cd-4ab3-8d79-e885a6463265","Type":"ContainerStarted","Data":"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"}
Jul 28 18:36:21.560959 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:21.560606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:21.582063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:21.582009    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" podStartSLOduration=3.054976119 podStartE2EDuration="3.581991137s" totalImagesPullingTime="527.015018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:36:21.577068014 +0000 UTC m=+2361.147130846" watchObservedRunningTime="2026-07-28 18:36:21.581991137 +0000 UTC m=+2361.152054007"
Jul 28 18:36:27.570574 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:27.570534    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:36:31.562934 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:31.562904    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:36:31.563446 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:36:31.563195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:38:15.575913 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.575876    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:38:15.576552 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.576184    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" podUID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" containerName="maas-api" containerID="cri-o://f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33" gracePeriod=30
Jul 28 18:38:15.634751 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.634712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:38:15.635063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.635012    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" podUID="6b4900c5-ba80-463e-b105-3eb92db4fe86" containerName="payload-processing" containerID="cri-o://22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae" gracePeriod=30
Jul 28 18:38:15.640614 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.640582    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:38:15.640876 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.640828    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" podUID="7766dc40-c248-4bde-ba16-8bb93adb2717" containerName="payload-pre-processing" containerID="cri-o://6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea" gracePeriod=30
Jul 28 18:38:15.943155 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.943123    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:38:15.946514 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:15.946483    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:38:16.031618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.031591    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:38:16.115373 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.115253    2581 generic.go:356] "Generic (PLEG): container finished" podID="6b4900c5-ba80-463e-b105-3eb92db4fe86" containerID="22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae" exitCode=0
Jul 28 18:38:16.115373 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.115315    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"
Jul 28 18:38:16.115373 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.115354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" event={"ID":"6b4900c5-ba80-463e-b105-3eb92db4fe86","Type":"ContainerDied","Data":"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"}
Jul 28 18:38:16.115641 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.115407    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n" event={"ID":"6b4900c5-ba80-463e-b105-3eb92db4fe86","Type":"ContainerDied","Data":"12a740143084c61d8710697d5e7d8af87af8975e6d6f4ae433ce2d0da5b33f52"}
Jul 28 18:38:16.115641 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.115431    2581 scope.go:117] "RemoveContainer" containerID="22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"
Jul 28 18:38:16.116731 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.116704    2581 generic.go:356] "Generic (PLEG): container finished" podID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" containerID="f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33" exitCode=0
Jul 28 18:38:16.116853 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.116752    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"
Jul 28 18:38:16.116853 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.116783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" event={"ID":"fdeda853-0c7e-4201-ab8f-9704e4e102eb","Type":"ContainerDied","Data":"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"}
Jul 28 18:38:16.116853 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.116841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv" event={"ID":"fdeda853-0c7e-4201-ab8f-9704e4e102eb","Type":"ContainerDied","Data":"a0440bf4ad0c687c400c696cba7bc3d0d95f9b0c0ef866dcf37e3986c2c3419b"}
Jul 28 18:38:16.118121 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.118101    2581 generic.go:356] "Generic (PLEG): container finished" podID="7766dc40-c248-4bde-ba16-8bb93adb2717" containerID="6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea" exitCode=0
Jul 28 18:38:16.118238 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.118125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" event={"ID":"7766dc40-c248-4bde-ba16-8bb93adb2717","Type":"ContainerDied","Data":"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"}
Jul 28 18:38:16.118238 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.118141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"
Jul 28 18:38:16.118238 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.118166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28" event={"ID":"7766dc40-c248-4bde-ba16-8bb93adb2717","Type":"ContainerDied","Data":"e86d94cec0a929b55a1f056baea85fb3a84b3828cceafbb739fa10b89de1b578"}
Jul 28 18:38:16.121419 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121399    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume\") pod \"6b4900c5-ba80-463e-b105-3eb92db4fe86\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") "
Jul 28 18:38:16.121507 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume\") pod \"7766dc40-c248-4bde-ba16-8bb93adb2717\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") "
Jul 28 18:38:16.121507 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121473    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrp8t\" (UniqueName: \"kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t\") pod \"6b4900c5-ba80-463e-b105-3eb92db4fe86\" (UID: \"6b4900c5-ba80-463e-b105-3eb92db4fe86\") "
Jul 28 18:38:16.121507 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121497    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q9ld\" (UniqueName: \"kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld\") pod \"7766dc40-c248-4bde-ba16-8bb93adb2717\" (UID: \"7766dc40-c248-4bde-ba16-8bb93adb2717\") "
Jul 28 18:38:16.121973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121945    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b4900c5-ba80-463e-b105-3eb92db4fe86" (UID: "6b4900c5-ba80-463e-b105-3eb92db4fe86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:16.121973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.121971    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7766dc40-c248-4bde-ba16-8bb93adb2717" (UID: "7766dc40-c248-4bde-ba16-8bb93adb2717"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:16.123537 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.123514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t" (OuterVolumeSpecName: "kube-api-access-lrp8t") pod "6b4900c5-ba80-463e-b105-3eb92db4fe86" (UID: "6b4900c5-ba80-463e-b105-3eb92db4fe86"). InnerVolumeSpecName "kube-api-access-lrp8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:16.123611 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.123554    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld" (OuterVolumeSpecName: "kube-api-access-9q9ld") pod "7766dc40-c248-4bde-ba16-8bb93adb2717" (UID: "7766dc40-c248-4bde-ba16-8bb93adb2717"). InnerVolumeSpecName "kube-api-access-9q9ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:16.124450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.124435    2581 scope.go:117] "RemoveContainer" containerID="22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"
Jul 28 18:38:16.124697 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:16.124667    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae\": container with ID starting with 22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae not found: ID does not exist" containerID="22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"
Jul 28 18:38:16.124796 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.124694    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae"} err="failed to get container status \"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae\": rpc error: code = NotFound desc = could not find container \"22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae\": container with ID starting with 22a25cec6e6ff2298fd28c2a8f1761fbb4d65263921077ae9d9ec8d9e19c40ae not found: ID does not exist"
Jul 28 18:38:16.124796 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.124711    2581 scope.go:117] "RemoveContainer" containerID="f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"
Jul 28 18:38:16.132500 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.132484    2581 scope.go:117] "RemoveContainer" containerID="f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"
Jul 28 18:38:16.132738 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:16.132717    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33\": container with ID starting with f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33 not found: ID does not exist" containerID="f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"
Jul 28 18:38:16.132838 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.132745    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33"} err="failed to get container status \"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33\": rpc error: code = NotFound desc = could not find container \"f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33\": container with ID starting with f3102434949a57049c25de44ed55e42e60bd86e0e9441a252a7e90a472743b33 not found: ID does not exist"
Jul 28 18:38:16.132838 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.132761    2581 scope.go:117] "RemoveContainer" containerID="6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"
Jul 28 18:38:16.140242 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.140228    2581 scope.go:117] "RemoveContainer" containerID="6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"
Jul 28 18:38:16.140495 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:16.140478    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea\": container with ID starting with 6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea not found: ID does not exist" containerID="6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"
Jul 28 18:38:16.140558 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.140501    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea"} err="failed to get container status \"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea\": rpc error: code = NotFound desc = could not find container \"6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea\": container with ID starting with 6f039983aa6c170d4108c1e316e780a2324a2aa90e08d8331138595477429bea not found: ID does not exist"
Jul 28 18:38:16.222773 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.222744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls\") pod \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") "
Jul 28 18:38:16.222948 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.222799    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wptcx\" (UniqueName: \"kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx\") pod \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") "
Jul 28 18:38:16.222948 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.222847    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca\") pod \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\" (UID: \"fdeda853-0c7e-4201-ab8f-9704e4e102eb\") "
Jul 28 18:38:16.223180 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.223154    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrp8t\" (UniqueName: \"kubernetes.io/projected/6b4900c5-ba80-463e-b105-3eb92db4fe86-kube-api-access-lrp8t\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.223180 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.223182    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q9ld\" (UniqueName: \"kubernetes.io/projected/7766dc40-c248-4bde-ba16-8bb93adb2717-kube-api-access-9q9ld\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.223375 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.223197    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b4900c5-ba80-463e-b105-3eb92db4fe86-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.223375 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.223212    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7766dc40-c248-4bde-ba16-8bb93adb2717-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.223375 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.223302    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdeda853-0c7e-4201-ab8f-9704e4e102eb" (UID: "fdeda853-0c7e-4201-ab8f-9704e4e102eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:16.224809 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.224788    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdeda853-0c7e-4201-ab8f-9704e4e102eb" (UID: "fdeda853-0c7e-4201-ab8f-9704e4e102eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:38:16.224885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.224865    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx" (OuterVolumeSpecName: "kube-api-access-wptcx") pod "fdeda853-0c7e-4201-ab8f-9704e4e102eb" (UID: "fdeda853-0c7e-4201-ab8f-9704e4e102eb"). InnerVolumeSpecName "kube-api-access-wptcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:16.323931 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.323892    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdeda853-0c7e-4201-ab8f-9704e4e102eb-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.323931 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.323924    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdeda853-0c7e-4201-ab8f-9704e4e102eb-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.323931 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.323933    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wptcx\" (UniqueName: \"kubernetes.io/projected/fdeda853-0c7e-4201-ab8f-9704e4e102eb-kube-api-access-wptcx\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:16.442234 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.442205    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:38:16.446103 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.446079    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b228ad-6cf7bd669d-fjdtv"]
Jul 28 18:38:16.456346 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.456322    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:38:16.460643 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.460617    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b228ad-6dd9dcd684-4gd4n"]
Jul 28 18:38:16.471011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.470987    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:38:16.475510 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.475487    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b228ad-6c8fc4f57b-bjn28"]
Jul 28 18:38:16.964262 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.964183    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4900c5-ba80-463e-b105-3eb92db4fe86" path="/var/lib/kubelet/pods/6b4900c5-ba80-463e-b105-3eb92db4fe86/volumes"
Jul 28 18:38:16.964634 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.964576    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7766dc40-c248-4bde-ba16-8bb93adb2717" path="/var/lib/kubelet/pods/7766dc40-c248-4bde-ba16-8bb93adb2717/volumes"
Jul 28 18:38:16.964934 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:16.964920    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" path="/var/lib/kubelet/pods/fdeda853-0c7e-4201-ab8f-9704e4e102eb/volumes"
Jul 28 18:38:33.273144 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.273110    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:38:33.273913 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.273861    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" podUID="70534586-b6cd-4ab3-8d79-e885a6463265" containerName="maas-api" containerID="cri-o://ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338" gracePeriod=30
Jul 28 18:38:33.327849 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.327811    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:38:33.328167 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.328084    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" podUID="067dc2a6-14eb-4b77-8768-b480968b069e" containerName="payload-processing" containerID="cri-o://245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9" gracePeriod=30
Jul 28 18:38:33.335541 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.335367    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:38:33.335702 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.335602    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" podUID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" containerName="payload-pre-processing" containerID="cri-o://996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9" gracePeriod=30
Jul 28 18:38:33.615684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.615657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:38:33.628586 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.628567    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:38:33.636051 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.636024    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:38:33.677410 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677380    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca\") pod \"70534586-b6cd-4ab3-8d79-e885a6463265\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") "
Jul 28 18:38:33.677561 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdcm4\" (UniqueName: \"kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4\") pod \"70534586-b6cd-4ab3-8d79-e885a6463265\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") "
Jul 28 18:38:33.677561 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume\") pod \"067dc2a6-14eb-4b77-8768-b480968b069e\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") "
Jul 28 18:38:33.677561 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume\") pod \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") "
Jul 28 18:38:33.677561 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677549    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") pod \"70534586-b6cd-4ab3-8d79-e885a6463265\" (UID: \"70534586-b6cd-4ab3-8d79-e885a6463265\") "
Jul 28 18:38:33.677806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677573    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sshw6\" (UniqueName: \"kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6\") pod \"067dc2a6-14eb-4b77-8768-b480968b069e\" (UID: \"067dc2a6-14eb-4b77-8768-b480968b069e\") "
Jul 28 18:38:33.677806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677608    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvkhg\" (UniqueName: \"kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg\") pod \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\" (UID: \"7aad389a-e02f-4887-b05d-7b5d3cb9d514\") "
Jul 28 18:38:33.677907 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70534586-b6cd-4ab3-8d79-e885a6463265" (UID: "70534586-b6cd-4ab3-8d79-e885a6463265"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:33.677962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7aad389a-e02f-4887-b05d-7b5d3cb9d514" (UID: "7aad389a-e02f-4887-b05d-7b5d3cb9d514"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:33.677962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.677917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "067dc2a6-14eb-4b77-8768-b480968b069e" (UID: "067dc2a6-14eb-4b77-8768-b480968b069e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:38:33.678083 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.678060    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70534586-b6cd-4ab3-8d79-e885a6463265-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.678143 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.678086    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/067dc2a6-14eb-4b77-8768-b480968b069e-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.678143 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.678100    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7aad389a-e02f-4887-b05d-7b5d3cb9d514-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.679826 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.679792    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg" (OuterVolumeSpecName: "kube-api-access-gvkhg") pod "7aad389a-e02f-4887-b05d-7b5d3cb9d514" (UID: "7aad389a-e02f-4887-b05d-7b5d3cb9d514"). InnerVolumeSpecName "kube-api-access-gvkhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:33.679931 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.679877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6" (OuterVolumeSpecName: "kube-api-access-sshw6") pod "067dc2a6-14eb-4b77-8768-b480968b069e" (UID: "067dc2a6-14eb-4b77-8768-b480968b069e"). InnerVolumeSpecName "kube-api-access-sshw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:33.680068 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.680046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4" (OuterVolumeSpecName: "kube-api-access-fdcm4") pod "70534586-b6cd-4ab3-8d79-e885a6463265" (UID: "70534586-b6cd-4ab3-8d79-e885a6463265"). InnerVolumeSpecName "kube-api-access-fdcm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:38:33.680140 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.680111    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70534586-b6cd-4ab3-8d79-e885a6463265" (UID: "70534586-b6cd-4ab3-8d79-e885a6463265"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:38:33.778973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.778933    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdcm4\" (UniqueName: \"kubernetes.io/projected/70534586-b6cd-4ab3-8d79-e885a6463265-kube-api-access-fdcm4\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.778973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.778963    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70534586-b6cd-4ab3-8d79-e885a6463265-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.778973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.778973    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sshw6\" (UniqueName: \"kubernetes.io/projected/067dc2a6-14eb-4b77-8768-b480968b069e-kube-api-access-sshw6\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:33.778973 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:33.778982    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvkhg\" (UniqueName: \"kubernetes.io/projected/7aad389a-e02f-4887-b05d-7b5d3cb9d514-kube-api-access-gvkhg\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:38:34.212690 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.212650    2581 generic.go:356] "Generic (PLEG): container finished" podID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" containerID="996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9" exitCode=0
Jul 28 18:38:34.212690 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.212694    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"
Jul 28 18:38:34.212955 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.212752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" event={"ID":"7aad389a-e02f-4887-b05d-7b5d3cb9d514","Type":"ContainerDied","Data":"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"}
Jul 28 18:38:34.212955 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.212823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d" event={"ID":"7aad389a-e02f-4887-b05d-7b5d3cb9d514","Type":"ContainerDied","Data":"068c98cc2f0c630d1b66c17f0fec8747be5e35f083a6c2a7f916a461cc45bc07"}
Jul 28 18:38:34.212955 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.212856    2581 scope.go:117] "RemoveContainer" containerID="996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"
Jul 28 18:38:34.214305 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.214261    2581 generic.go:356] "Generic (PLEG): container finished" podID="067dc2a6-14eb-4b77-8768-b480968b069e" containerID="245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9" exitCode=0
Jul 28 18:38:34.214450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.214327    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"
Jul 28 18:38:34.214450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.214358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" event={"ID":"067dc2a6-14eb-4b77-8768-b480968b069e","Type":"ContainerDied","Data":"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"}
Jul 28 18:38:34.214450 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.214395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd" event={"ID":"067dc2a6-14eb-4b77-8768-b480968b069e","Type":"ContainerDied","Data":"4ec313e53e5094097e3218c8d889ca05fb320d86481ebdeeddcd92eed5de2177"}
Jul 28 18:38:34.216118 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.216089    2581 generic.go:356] "Generic (PLEG): container finished" podID="70534586-b6cd-4ab3-8d79-e885a6463265" containerID="ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338" exitCode=0
Jul 28 18:38:34.216250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.216140    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"
Jul 28 18:38:34.216250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.216180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" event={"ID":"70534586-b6cd-4ab3-8d79-e885a6463265","Type":"ContainerDied","Data":"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"}
Jul 28 18:38:34.216250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.216221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r" event={"ID":"70534586-b6cd-4ab3-8d79-e885a6463265","Type":"ContainerDied","Data":"fe69092b7c529b8a011aec960a26990bf7e60b9e5948a1ea9d2fdd3685db8a75"}
Jul 28 18:38:34.223137 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.223115    2581 scope.go:117] "RemoveContainer" containerID="996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"
Jul 28 18:38:34.223547 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:34.223501    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9\": container with ID starting with 996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9 not found: ID does not exist" containerID="996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"
Jul 28 18:38:34.223668 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.223539    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9"} err="failed to get container status \"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9\": rpc error: code = NotFound desc = could not find container \"996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9\": container with ID starting with 996ab7a63d2c119484366465dd25d785c96091db53715c4fbb22c8352e9e78e9 not found: ID does not exist"
Jul 28 18:38:34.223668 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.223565    2581 scope.go:117] "RemoveContainer" containerID="245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"
Jul 28 18:38:34.233773 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.233754    2581 scope.go:117] "RemoveContainer" containerID="245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"
Jul 28 18:38:34.234092 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:34.234073    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9\": container with ID starting with 245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9 not found: ID does not exist" containerID="245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"
Jul 28 18:38:34.234149 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.234101    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9"} err="failed to get container status \"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9\": rpc error: code = NotFound desc = could not find container \"245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9\": container with ID starting with 245e2f342a7fcf65d0bb35e307784cb3f620f7b035f3343ea0e63728eec051c9 not found: ID does not exist"
Jul 28 18:38:34.234149 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.234120    2581 scope.go:117] "RemoveContainer" containerID="ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"
Jul 28 18:38:34.239164 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.239119    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:38:34.241470 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.241448    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-210335-bfc47c45b-sj77d"]
Jul 28 18:38:34.243471 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.243456    2581 scope.go:117] "RemoveContainer" containerID="ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"
Jul 28 18:38:34.243788 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:34.243769    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338\": container with ID starting with ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338 not found: ID does not exist" containerID="ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"
Jul 28 18:38:34.243840 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.243796    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338"} err="failed to get container status \"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338\": rpc error: code = NotFound desc = could not find container \"ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338\": container with ID starting with ef8ea6d7fac27585b5f705cc4110f079a14b5f5023947187d810defd5697e338 not found: ID does not exist"
Jul 28 18:38:34.250859 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.250835    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:38:34.255219 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.255197    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-210335-9b79b4fd4-pnwjd"]
Jul 28 18:38:34.268836 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.268812    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:38:34.274809 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.274772    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-210335-7b6cd87bb-4c64r"]
Jul 28 18:38:34.963623 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.963590    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="067dc2a6-14eb-4b77-8768-b480968b069e" path="/var/lib/kubelet/pods/067dc2a6-14eb-4b77-8768-b480968b069e/volumes"
Jul 28 18:38:34.963991 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.963979    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70534586-b6cd-4ab3-8d79-e885a6463265" path="/var/lib/kubelet/pods/70534586-b6cd-4ab3-8d79-e885a6463265/volumes"
Jul 28 18:38:34.964312 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:34.964299    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" path="/var/lib/kubelet/pods/7aad389a-e02f-4887-b05d-7b5d3cb9d514/volumes"
Jul 28 18:38:43.899699 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.899663    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900009    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7766dc40-c248-4bde-ba16-8bb93adb2717" containerName="payload-pre-processing"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900024    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7766dc40-c248-4bde-ba16-8bb93adb2717" containerName="payload-pre-processing"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900036    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70534586-b6cd-4ab3-8d79-e885a6463265" containerName="maas-api"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900044    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="70534586-b6cd-4ab3-8d79-e885a6463265" containerName="maas-api"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900059    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="067dc2a6-14eb-4b77-8768-b480968b069e" containerName="payload-processing"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="067dc2a6-14eb-4b77-8768-b480968b069e" containerName="payload-processing"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900094    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" containerName="maas-api"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900103    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" containerName="maas-api"
Jul 28 18:38:43.900106 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900112    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" containerName="payload-pre-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900119    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" containerName="payload-pre-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900131    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4900c5-ba80-463e-b105-3eb92db4fe86" containerName="payload-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900138    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4900c5-ba80-463e-b105-3eb92db4fe86" containerName="payload-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900204    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b4900c5-ba80-463e-b105-3eb92db4fe86" containerName="payload-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aad389a-e02f-4887-b05d-7b5d3cb9d514" containerName="payload-pre-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900220    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7766dc40-c248-4bde-ba16-8bb93adb2717" containerName="payload-pre-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900226    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="067dc2a6-14eb-4b77-8768-b480968b069e" containerName="payload-processing"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900231    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="70534586-b6cd-4ab3-8d79-e885a6463265" containerName="maas-api"
Jul 28 18:38:43.900438 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.900257    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdeda853-0c7e-4201-ab8f-9704e4e102eb" containerName="maas-api"
Jul 28 18:38:43.903564 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.903541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:43.905679 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.905653    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a7357c-serving-cert\""
Jul 28 18:38:43.915724 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.915701    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:38:43.976554 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.976497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm7hd\" (UniqueName: \"kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:43.976759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.976583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:43.976759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:43.976698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.077985 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.077941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.078175 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.078082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.078175 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:44.078147    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-serving-cert: secret "maas-api-e2e-api-a-a7357c-serving-cert" not found
Jul 28 18:38:44.078350 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:38:44.078223    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls podName:6d40c625-f922-4a21-a35f-510366121dbd nodeName:}" failed. No retries permitted until 2026-07-28 18:38:44.578201813 +0000 UTC m=+2504.148264625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls") pod "maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" (UID: "6d40c625-f922-4a21-a35f-510366121dbd") : secret "maas-api-e2e-api-a-a7357c-serving-cert" not found
Jul 28 18:38:44.078350 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.078304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm7hd\" (UniqueName: \"kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.078838 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.078817    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.089568 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.089533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm7hd\" (UniqueName: \"kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.583905 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.583862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.586419 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.586396    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") pod \"maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.592555 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.592525    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:38:44.596494 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.596473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.598682 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.598657    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a7357c-dockercfg-f9z4c\""
Jul 28 18:38:44.598849 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.598828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a7357c\""
Jul 28 18:38:44.607011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.606982    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:38:44.614353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.614322    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:38:44.617652 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.617629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.632648 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.632621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:38:44.685150 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.685109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2l7\" (UniqueName: \"kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.685353 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.685184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.786466 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.786420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.786676 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.786484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf2qg\" (UniqueName: \"kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.786676 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.786524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2l7\" (UniqueName: \"kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.786676 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.786552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.787167 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.787147    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.795766 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.795739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2l7\" (UniqueName: \"kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7\") pod \"payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.814731 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.814697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:44.887851 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.887816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf2qg\" (UniqueName: \"kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.888040 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.887901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.888881 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.888855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.902721 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.902660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf2qg\" (UniqueName: \"kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg\") pod \"payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") " pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.908626 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.908599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:44.929477 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.929442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:44.970309 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.970254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:38:44.974379 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:38:44.974334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d40c625_f922_4a21_a35f_510366121dbd.slice/crio-8c3cb764b0c24bb7202916df3589b2e87b166bb2d0f42a7f45803e8feab6dfb3 WatchSource:0}: Error finding container 8c3cb764b0c24bb7202916df3589b2e87b166bb2d0f42a7f45803e8feab6dfb3: Status 404 returned error can't find the container with id 8c3cb764b0c24bb7202916df3589b2e87b166bb2d0f42a7f45803e8feab6dfb3
Jul 28 18:38:44.975596 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:44.975559    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:38:45.096134 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.096091    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:38:45.096620 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:38:45.096572    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db368b1_809e_4d0f_bb6e_51fd10709efb.slice/crio-9e658eaaa4da92623e577bb4acba4ca2a80269a4ce25cdbfea9f58422a9b22f7 WatchSource:0}: Error finding container 9e658eaaa4da92623e577bb4acba4ca2a80269a4ce25cdbfea9f58422a9b22f7: Status 404 returned error can't find the container with id 9e658eaaa4da92623e577bb4acba4ca2a80269a4ce25cdbfea9f58422a9b22f7
Jul 28 18:38:45.124602 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.124577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:38:45.133530 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:38:45.133489    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511c74af_ccc4_4d1c_959c_abe890548598.slice/crio-87e1a9b26eca377565d96ccabbf5cd90206f5e2c5e063d562fd91d30d3fdbd00 WatchSource:0}: Error finding container 87e1a9b26eca377565d96ccabbf5cd90206f5e2c5e063d562fd91d30d3fdbd00: Status 404 returned error can't find the container with id 87e1a9b26eca377565d96ccabbf5cd90206f5e2c5e063d562fd91d30d3fdbd00
Jul 28 18:38:45.272711 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.272669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" event={"ID":"6d40c625-f922-4a21-a35f-510366121dbd","Type":"ContainerStarted","Data":"8c3cb764b0c24bb7202916df3589b2e87b166bb2d0f42a7f45803e8feab6dfb3"}
Jul 28 18:38:45.274832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.274799    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" event={"ID":"5db368b1-809e-4d0f-bb6e-51fd10709efb","Type":"ContainerStarted","Data":"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"}
Jul 28 18:38:45.274989 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.274853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" event={"ID":"5db368b1-809e-4d0f-bb6e-51fd10709efb","Type":"ContainerStarted","Data":"9e658eaaa4da92623e577bb4acba4ca2a80269a4ce25cdbfea9f58422a9b22f7"}
Jul 28 18:38:45.274989 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.274902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:38:45.276790 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.276762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" event={"ID":"511c74af-ccc4-4d1c-959c-abe890548598","Type":"ContainerStarted","Data":"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"}
Jul 28 18:38:45.276914 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.276813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" event={"ID":"511c74af-ccc4-4d1c-959c-abe890548598","Type":"ContainerStarted","Data":"87e1a9b26eca377565d96ccabbf5cd90206f5e2c5e063d562fd91d30d3fdbd00"}
Jul 28 18:38:45.276914 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.276882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:38:45.298477 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.298424    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" podStartSLOduration=1.298404969 podStartE2EDuration="1.298404969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:45.29729367 +0000 UTC m=+2504.867356494" watchObservedRunningTime="2026-07-28 18:38:45.298404969 +0000 UTC m=+2504.868467800"
Jul 28 18:38:45.318332 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:45.318248    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" podStartSLOduration=1.318233357 podStartE2EDuration="1.318233357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:45.316420714 +0000 UTC m=+2504.886483536" watchObservedRunningTime="2026-07-28 18:38:45.318233357 +0000 UTC m=+2504.888296187"
Jul 28 18:38:46.284849 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:46.284772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" event={"ID":"6d40c625-f922-4a21-a35f-510366121dbd","Type":"ContainerStarted","Data":"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"}
Jul 28 18:38:46.304248 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:46.304194    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" podStartSLOduration=2.824521811 podStartE2EDuration="3.304177071s" totalImagesPullingTime="479.65526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:38:46.302419073 +0000 UTC m=+2505.872481903" watchObservedRunningTime="2026-07-28 18:38:46.304177071 +0000 UTC m=+2505.874239947"
Jul 28 18:38:47.289685 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:47.289647    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:53.300308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:53.300198    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:38:56.286871 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:56.286826    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.57:9004: connect: connection refused"
Jul 28 18:38:56.287313 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:38:56.287293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:39:04.909225 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:04.909174    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.57:9004: connect: connection refused"
Jul 28 18:39:06.286401 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:06.286373    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:39:13.072577 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.072534    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:39:13.076159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.076133    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.079580 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.079549    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-51a15d-serving-cert\""
Jul 28 18:39:13.144134 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.144099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.144134 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.144143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htzg6\" (UniqueName: \"kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.144393 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.144185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.144393 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.144254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:39:13.245852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.245811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.246050 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.245864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htzg6\" (UniqueName: \"kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.246050 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.245907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.246050 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:39:13.245989    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-serving-cert: secret "maas-api-e2e-api-b-51a15d-serving-cert" not found
Jul 28 18:39:13.246214 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:39:13.246161    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls podName:a409c864-b3b1-4cf3-a562-3b48ed1ac50c nodeName:}" failed. No retries permitted until 2026-07-28 18:39:13.746138681 +0000 UTC m=+2533.316201501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls") pod "maas-api-e2e-api-b-51a15d-5584595454-rwlsc" (UID: "a409c864-b3b1-4cf3-a562-3b48ed1ac50c") : secret "maas-api-e2e-api-b-51a15d-serving-cert" not found
Jul 28 18:39:13.246605 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.246586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.278036 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.278007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htzg6\" (UniqueName: \"kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.752324 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:13.752261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:13.752502 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:39:13.752418    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-serving-cert: secret "maas-api-e2e-api-b-51a15d-serving-cert" not found
Jul 28 18:39:13.752502 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:39:13.752484    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls podName:a409c864-b3b1-4cf3-a562-3b48ed1ac50c nodeName:}" failed. No retries permitted until 2026-07-28 18:39:14.752469235 +0000 UTC m=+2534.322532044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls") pod "maas-api-e2e-api-b-51a15d-5584595454-rwlsc" (UID: "a409c864-b3b1-4cf3-a562-3b48ed1ac50c") : secret "maas-api-e2e-api-b-51a15d-serving-cert" not found
Jul 28 18:39:14.324957 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.324919    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:39:14.328491 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.328467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.332121 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.332098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-51a15d-dockercfg-5hbz7\""
Jul 28 18:39:14.332319 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.332302    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-51a15d\""
Jul 28 18:39:14.345632 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.345603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:39:14.458528 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.458493    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:39:14.460429 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.460394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.460575 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.460501    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7nd5\" (UniqueName: \"kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.461762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.461748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.474996 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.474967    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:39:14.561236 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.561187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7nd5\" (UniqueName: \"kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.561458 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.561246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.561458 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.561353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp2w7\" (UniqueName: \"kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.561568 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.561503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.562185 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.562155    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.569620 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.569594    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7nd5\" (UniqueName: \"kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5\") pod \"payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.638987 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.638896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:14.662377 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.662340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.662543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.662430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp2w7\" (UniqueName: \"kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.663028 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.663007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.670837 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.670812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp2w7\" (UniqueName: \"kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7\") pod \"payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") " pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.763225 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.763187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:14.765801 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.765778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"maas-api-e2e-api-b-51a15d-5584595454-rwlsc\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:14.772565 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.772535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:14.790173 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.790147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:39:14.791877 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:39:14.791855    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4eba21_e660_4e90_a19e_2d0dc7f0516e.slice/crio-58196b1897cc173c2f0b6f6b0ed7ba7c7aa549fcfc1404129a7166a46dc5d3e5 WatchSource:0}: Error finding container 58196b1897cc173c2f0b6f6b0ed7ba7c7aa549fcfc1404129a7166a46dc5d3e5: Status 404 returned error can't find the container with id 58196b1897cc173c2f0b6f6b0ed7ba7c7aa549fcfc1404129a7166a46dc5d3e5
Jul 28 18:39:14.890978 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:14.890896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:15.039703 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.039665    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:39:15.041076 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:39:15.041046    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda409c864_b3b1_4cf3_a562_3b48ed1ac50c.slice/crio-a8a1b9930480bc968daa02329be05caedb4177b7058405f865fb0957eb86b4fc WatchSource:0}: Error finding container a8a1b9930480bc968daa02329be05caedb4177b7058405f865fb0957eb86b4fc: Status 404 returned error can't find the container with id a8a1b9930480bc968daa02329be05caedb4177b7058405f865fb0957eb86b4fc
Jul 28 18:39:15.128794 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.128762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:39:15.131763 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:39:15.131733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e1f043_bdab_43aa_be36_99b57a0c3ce5.slice/crio-6e9ddaa443af50ff81993a7ff010401f6b97732b174539dc4c409a27f9172c34 WatchSource:0}: Error finding container 6e9ddaa443af50ff81993a7ff010401f6b97732b174539dc4c409a27f9172c34: Status 404 returned error can't find the container with id 6e9ddaa443af50ff81993a7ff010401f6b97732b174539dc4c409a27f9172c34
Jul 28 18:39:15.426585 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.426543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" event={"ID":"34e1f043-bdab-43aa-be36-99b57a0c3ce5","Type":"ContainerStarted","Data":"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"}
Jul 28 18:39:15.426975 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.426684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:39:15.426975 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.426715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" event={"ID":"34e1f043-bdab-43aa-be36-99b57a0c3ce5","Type":"ContainerStarted","Data":"6e9ddaa443af50ff81993a7ff010401f6b97732b174539dc4c409a27f9172c34"}
Jul 28 18:39:15.428055 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.428032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" event={"ID":"2c4eba21-e660-4e90-a19e-2d0dc7f0516e","Type":"ContainerStarted","Data":"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"}
Jul 28 18:39:15.428148 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.428071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" event={"ID":"2c4eba21-e660-4e90-a19e-2d0dc7f0516e","Type":"ContainerStarted","Data":"58196b1897cc173c2f0b6f6b0ed7ba7c7aa549fcfc1404129a7166a46dc5d3e5"}
Jul 28 18:39:15.428148 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.428116    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:15.429369 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.429342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" event={"ID":"a409c864-b3b1-4cf3-a562-3b48ed1ac50c","Type":"ContainerStarted","Data":"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"}
Jul 28 18:39:15.429478 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.429379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" event={"ID":"a409c864-b3b1-4cf3-a562-3b48ed1ac50c","Type":"ContainerStarted","Data":"a8a1b9930480bc968daa02329be05caedb4177b7058405f865fb0957eb86b4fc"}
Jul 28 18:39:15.429478 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.429463    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:15.443228 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.443186    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" podStartSLOduration=1.443171305 podStartE2EDuration="1.443171305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:39:15.442046482 +0000 UTC m=+2535.012115213" watchObservedRunningTime="2026-07-28 18:39:15.443171305 +0000 UTC m=+2535.013234135"
Jul 28 18:39:15.465288 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.461514    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" podStartSLOduration=1.4614953050000001 podStartE2EDuration="1.461495305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:39:15.45779908 +0000 UTC m=+2535.027861913" watchObservedRunningTime="2026-07-28 18:39:15.461495305 +0000 UTC m=+2535.031558139"
Jul 28 18:39:15.478315 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:15.478045    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" podStartSLOduration=2.168472196 podStartE2EDuration="2.478023877s" totalImagesPullingTime="309.551681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:39:15.474744912 +0000 UTC m=+2535.044807745" watchObservedRunningTime="2026-07-28 18:39:15.478023877 +0000 UTC m=+2535.048086711"
Jul 28 18:39:21.441426 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:21.441387    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:39:26.436875 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:26.436844    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:39:26.437395 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:39:26.436927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:40:01.794345 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.794310    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:40:01.794889 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.794513    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" podUID="6d40c625-f922-4a21-a35f-510366121dbd" containerName="maas-api" containerID="cri-o://46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01" gracePeriod=30
Jul 28 18:40:01.949985 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.949947    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:40:01.950250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.950219    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" podUID="511c74af-ccc4-4d1c-959c-abe890548598" containerName="payload-processing" containerID="cri-o://d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c" gracePeriod=30
Jul 28 18:40:01.964921 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.964873    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:40:01.973914 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:01.965323    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing" containerID="cri-o://fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b" gracePeriod=30
Jul 28 18:40:02.101012 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.100986    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:40:02.223189 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.223116    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm7hd\" (UniqueName: \"kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd\") pod \"6d40c625-f922-4a21-a35f-510366121dbd\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") "
Jul 28 18:40:02.223724 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.223260    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca\") pod \"6d40c625-f922-4a21-a35f-510366121dbd\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") "
Jul 28 18:40:02.223724 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.223326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") pod \"6d40c625-f922-4a21-a35f-510366121dbd\" (UID: \"6d40c625-f922-4a21-a35f-510366121dbd\") "
Jul 28 18:40:02.224093 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.224059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d40c625-f922-4a21-a35f-510366121dbd" (UID: "6d40c625-f922-4a21-a35f-510366121dbd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:02.227151 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.227122    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d40c625-f922-4a21-a35f-510366121dbd" (UID: "6d40c625-f922-4a21-a35f-510366121dbd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:40:02.229433 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.229406    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd" (OuterVolumeSpecName: "kube-api-access-cm7hd") pod "6d40c625-f922-4a21-a35f-510366121dbd" (UID: "6d40c625-f922-4a21-a35f-510366121dbd"). InnerVolumeSpecName "kube-api-access-cm7hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:02.252727 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.252702    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:40:02.255725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.255705    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:40:02.324834 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.324795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf2qg\" (UniqueName: \"kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg\") pod \"511c74af-ccc4-4d1c-959c-abe890548598\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") "
Jul 28 18:40:02.325005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.324882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume\") pod \"511c74af-ccc4-4d1c-959c-abe890548598\" (UID: \"511c74af-ccc4-4d1c-959c-abe890548598\") "
Jul 28 18:40:02.325005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.324915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf2l7\" (UniqueName: \"kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7\") pod \"5db368b1-809e-4d0f-bb6e-51fd10709efb\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") "
Jul 28 18:40:02.325005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.324941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume\") pod \"5db368b1-809e-4d0f-bb6e-51fd10709efb\" (UID: \"5db368b1-809e-4d0f-bb6e-51fd10709efb\") "
Jul 28 18:40:02.325193 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.325174    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d40c625-f922-4a21-a35f-510366121dbd-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.325240 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.325200    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d40c625-f922-4a21-a35f-510366121dbd-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.325240 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.325214    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm7hd\" (UniqueName: \"kubernetes.io/projected/6d40c625-f922-4a21-a35f-510366121dbd-kube-api-access-cm7hd\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.325402 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.325372    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "511c74af-ccc4-4d1c-959c-abe890548598" (UID: "511c74af-ccc4-4d1c-959c-abe890548598"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:02.325440 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.325410    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5db368b1-809e-4d0f-bb6e-51fd10709efb" (UID: "5db368b1-809e-4d0f-bb6e-51fd10709efb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:02.327107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.327071    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg" (OuterVolumeSpecName: "kube-api-access-cf2qg") pod "511c74af-ccc4-4d1c-959c-abe890548598" (UID: "511c74af-ccc4-4d1c-959c-abe890548598"). InnerVolumeSpecName "kube-api-access-cf2qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:02.327107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.327077    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7" (OuterVolumeSpecName: "kube-api-access-qf2l7") pod "5db368b1-809e-4d0f-bb6e-51fd10709efb" (UID: "5db368b1-809e-4d0f-bb6e-51fd10709efb"). InnerVolumeSpecName "kube-api-access-qf2l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:02.426011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.425900    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/511c74af-ccc4-4d1c-959c-abe890548598-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.426011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.425948    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf2l7\" (UniqueName: \"kubernetes.io/projected/5db368b1-809e-4d0f-bb6e-51fd10709efb-kube-api-access-qf2l7\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.426011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.425958    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db368b1-809e-4d0f-bb6e-51fd10709efb-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.426011 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.425967    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf2qg\" (UniqueName: \"kubernetes.io/projected/511c74af-ccc4-4d1c-959c-abe890548598-kube-api-access-cf2qg\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:02.660096 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.660062    2581 generic.go:356] "Generic (PLEG): container finished" podID="6d40c625-f922-4a21-a35f-510366121dbd" containerID="46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01" exitCode=0
Jul 28 18:40:02.660310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.660110    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"
Jul 28 18:40:02.660310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.660143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" event={"ID":"6d40c625-f922-4a21-a35f-510366121dbd","Type":"ContainerDied","Data":"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"}
Jul 28 18:40:02.660310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.660232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl" event={"ID":"6d40c625-f922-4a21-a35f-510366121dbd","Type":"ContainerDied","Data":"8c3cb764b0c24bb7202916df3589b2e87b166bb2d0f42a7f45803e8feab6dfb3"}
Jul 28 18:40:02.660310 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.660253    2581 scope.go:117] "RemoveContainer" containerID="46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"
Jul 28 18:40:02.661453 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.661430    2581 generic.go:356] "Generic (PLEG): container finished" podID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerID="fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b" exitCode=0
Jul 28 18:40:02.661548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.661466    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"
Jul 28 18:40:02.661608 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.661564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" event={"ID":"5db368b1-809e-4d0f-bb6e-51fd10709efb","Type":"ContainerDied","Data":"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"}
Jul 28 18:40:02.661668 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.661616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl" event={"ID":"5db368b1-809e-4d0f-bb6e-51fd10709efb","Type":"ContainerDied","Data":"9e658eaaa4da92623e577bb4acba4ca2a80269a4ce25cdbfea9f58422a9b22f7"}
Jul 28 18:40:02.662975 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.662952    2581 generic.go:356] "Generic (PLEG): container finished" podID="511c74af-ccc4-4d1c-959c-abe890548598" containerID="d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c" exitCode=0
Jul 28 18:40:02.663085 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.662994    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"
Jul 28 18:40:02.663085 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.663035    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" event={"ID":"511c74af-ccc4-4d1c-959c-abe890548598","Type":"ContainerDied","Data":"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"}
Jul 28 18:40:02.663185 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.663086    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm" event={"ID":"511c74af-ccc4-4d1c-959c-abe890548598","Type":"ContainerDied","Data":"87e1a9b26eca377565d96ccabbf5cd90206f5e2c5e063d562fd91d30d3fdbd00"}
Jul 28 18:40:02.669744 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.669613    2581 scope.go:117] "RemoveContainer" containerID="46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"
Jul 28 18:40:02.669955 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:02.669932    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01\": container with ID starting with 46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01 not found: ID does not exist" containerID="46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"
Jul 28 18:40:02.670027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.669963    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01"} err="failed to get container status \"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01\": rpc error: code = NotFound desc = could not find container \"46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01\": container with ID starting with 46a29a3fe108f4939e46bfe8065a1d83aa15aa35db4a610629da8c0ed4058f01 not found: ID does not exist"
Jul 28 18:40:02.670027 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.669979    2581 scope.go:117] "RemoveContainer" containerID="fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"
Jul 28 18:40:02.679489 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.679472    2581 scope.go:117] "RemoveContainer" containerID="fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"
Jul 28 18:40:02.679738 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:02.679723    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b\": container with ID starting with fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b not found: ID does not exist" containerID="fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"
Jul 28 18:40:02.679782 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.679744    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b"} err="failed to get container status \"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b\": rpc error: code = NotFound desc = could not find container \"fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b\": container with ID starting with fe79d493ac99b6210b99a38942dadc4666cdc151ceb2aab2b88a20adb6c1956b not found: ID does not exist"
Jul 28 18:40:02.679782 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.679759    2581 scope.go:117] "RemoveContainer" containerID="d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"
Jul 28 18:40:02.688870 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.688586    2581 scope.go:117] "RemoveContainer" containerID="d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"
Jul 28 18:40:02.689095 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:02.689071    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c\": container with ID starting with d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c not found: ID does not exist" containerID="d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"
Jul 28 18:40:02.689159 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.689103    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c"} err="failed to get container status \"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c\": rpc error: code = NotFound desc = could not find container \"d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c\": container with ID starting with d0c9c229cd91ccb1e8af4d734c780b35a674239a4444c9049a4ae63a2161386c not found: ID does not exist"
Jul 28 18:40:02.691473 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.691447    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:40:02.693153 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.693135    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7357c-7fd498cc5d-w5ctl"]
Jul 28 18:40:02.711490 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.711458    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:40:02.715022 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.714994    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a7357c-844d5cf944-z6mpm"]
Jul 28 18:40:02.728745 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.728709    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:40:02.732422 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.732391    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a7357c-65c787fdb8-wgfgl"]
Jul 28 18:40:02.964442 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.964358    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511c74af-ccc4-4d1c-959c-abe890548598" path="/var/lib/kubelet/pods/511c74af-ccc4-4d1c-959c-abe890548598/volumes"
Jul 28 18:40:02.964966 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.964731    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" path="/var/lib/kubelet/pods/5db368b1-809e-4d0f-bb6e-51fd10709efb/volumes"
Jul 28 18:40:02.965073 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:02.965059    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d40c625-f922-4a21-a35f-510366121dbd" path="/var/lib/kubelet/pods/6d40c625-f922-4a21-a35f-510366121dbd/volumes"
Jul 28 18:40:25.086390 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.086295    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:40:25.086864 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.086542    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" podUID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" containerName="maas-api" containerID="cri-o://46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9" gracePeriod=30
Jul 28 18:40:25.144817 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.144785    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:40:25.145021 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.144998    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" podUID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" containerName="payload-processing" containerID="cri-o://3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8" gracePeriod=30
Jul 28 18:40:25.158628 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.158593    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:40:25.158880 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.158858    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" podUID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" containerName="payload-pre-processing" containerID="cri-o://4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d" gracePeriod=30
Jul 28 18:40:25.457648 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.457622    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:40:25.462987 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.462967    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:40:25.480835 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.480809    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:40:25.552637 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552601    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7nd5\" (UniqueName: \"kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5\") pod \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") "
Jul 28 18:40:25.552832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552672    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") pod \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") "
Jul 28 18:40:25.552832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552709    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume\") pod \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\" (UID: \"2c4eba21-e660-4e90-a19e-2d0dc7f0516e\") "
Jul 28 18:40:25.552832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552733    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca\") pod \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") "
Jul 28 18:40:25.552832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume\") pod \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") "
Jul 28 18:40:25.552832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552782    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp2w7\" (UniqueName: \"kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7\") pod \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\" (UID: \"34e1f043-bdab-43aa-be36-99b57a0c3ce5\") "
Jul 28 18:40:25.553087 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.552852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htzg6\" (UniqueName: \"kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6\") pod \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\" (UID: \"a409c864-b3b1-4cf3-a562-3b48ed1ac50c\") "
Jul 28 18:40:25.553169 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.553135    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a409c864-b3b1-4cf3-a562-3b48ed1ac50c" (UID: "a409c864-b3b1-4cf3-a562-3b48ed1ac50c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:25.553237 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.553154    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34e1f043-bdab-43aa-be36-99b57a0c3ce5" (UID: "34e1f043-bdab-43aa-be36-99b57a0c3ce5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:25.553237 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.553150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c4eba21-e660-4e90-a19e-2d0dc7f0516e" (UID: "2c4eba21-e660-4e90-a19e-2d0dc7f0516e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:40:25.555121 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.555097    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6" (OuterVolumeSpecName: "kube-api-access-htzg6") pod "a409c864-b3b1-4cf3-a562-3b48ed1ac50c" (UID: "a409c864-b3b1-4cf3-a562-3b48ed1ac50c"). InnerVolumeSpecName "kube-api-access-htzg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:25.555246 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.555123    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7" (OuterVolumeSpecName: "kube-api-access-jp2w7") pod "34e1f043-bdab-43aa-be36-99b57a0c3ce5" (UID: "34e1f043-bdab-43aa-be36-99b57a0c3ce5"). InnerVolumeSpecName "kube-api-access-jp2w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:25.555308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.555253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a409c864-b3b1-4cf3-a562-3b48ed1ac50c" (UID: "a409c864-b3b1-4cf3-a562-3b48ed1ac50c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:40:25.555348 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.555260    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5" (OuterVolumeSpecName: "kube-api-access-x7nd5") pod "2c4eba21-e660-4e90-a19e-2d0dc7f0516e" (UID: "2c4eba21-e660-4e90-a19e-2d0dc7f0516e"). InnerVolumeSpecName "kube-api-access-x7nd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.653914    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htzg6\" (UniqueName: \"kubernetes.io/projected/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-kube-api-access-htzg6\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.653947    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7nd5\" (UniqueName: \"kubernetes.io/projected/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-kube-api-access-x7nd5\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.653962    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.653974    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4eba21-e660-4e90-a19e-2d0dc7f0516e-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.653987    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a409c864-b3b1-4cf3-a562-3b48ed1ac50c-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654005 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.654001    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34e1f043-bdab-43aa-be36-99b57a0c3ce5-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.654335 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.654014    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp2w7\" (UniqueName: \"kubernetes.io/projected/34e1f043-bdab-43aa-be36-99b57a0c3ce5-kube-api-access-jp2w7\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:40:25.774575 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.774539    2581 generic.go:356] "Generic (PLEG): container finished" podID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" containerID="3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8" exitCode=0
Jul 28 18:40:25.774749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.774579    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"
Jul 28 18:40:25.774749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.774624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" event={"ID":"34e1f043-bdab-43aa-be36-99b57a0c3ce5","Type":"ContainerDied","Data":"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"}
Jul 28 18:40:25.774749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.774682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh" event={"ID":"34e1f043-bdab-43aa-be36-99b57a0c3ce5","Type":"ContainerDied","Data":"6e9ddaa443af50ff81993a7ff010401f6b97732b174539dc4c409a27f9172c34"}
Jul 28 18:40:25.774749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.774712    2581 scope.go:117] "RemoveContainer" containerID="3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"
Jul 28 18:40:25.775933 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.775910    2581 generic.go:356] "Generic (PLEG): container finished" podID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" containerID="4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d" exitCode=0
Jul 28 18:40:25.776019 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.775961    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"
Jul 28 18:40:25.776019 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.775993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" event={"ID":"2c4eba21-e660-4e90-a19e-2d0dc7f0516e","Type":"ContainerDied","Data":"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"}
Jul 28 18:40:25.776120 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.776036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv" event={"ID":"2c4eba21-e660-4e90-a19e-2d0dc7f0516e","Type":"ContainerDied","Data":"58196b1897cc173c2f0b6f6b0ed7ba7c7aa549fcfc1404129a7166a46dc5d3e5"}
Jul 28 18:40:25.777338 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.777310    2581 generic.go:356] "Generic (PLEG): container finished" podID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" containerID="46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9" exitCode=0
Jul 28 18:40:25.777468 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.777351    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"
Jul 28 18:40:25.777468 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.777367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" event={"ID":"a409c864-b3b1-4cf3-a562-3b48ed1ac50c","Type":"ContainerDied","Data":"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"}
Jul 28 18:40:25.777468 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.777408    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc" event={"ID":"a409c864-b3b1-4cf3-a562-3b48ed1ac50c","Type":"ContainerDied","Data":"a8a1b9930480bc968daa02329be05caedb4177b7058405f865fb0957eb86b4fc"}
Jul 28 18:40:25.787652 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.787628    2581 scope.go:117] "RemoveContainer" containerID="3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"
Jul 28 18:40:25.792464 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:25.792261    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8\": container with ID starting with 3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8 not found: ID does not exist" containerID="3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"
Jul 28 18:40:25.792464 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.792332    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8"} err="failed to get container status \"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8\": rpc error: code = NotFound desc = could not find container \"3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8\": container with ID starting with 3f1d56e8759cdd6cb6e53daaf65c728d4ed6a23537cd91650a80217ea8b876b8 not found: ID does not exist"
Jul 28 18:40:25.792464 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.792359    2581 scope.go:117] "RemoveContainer" containerID="4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"
Jul 28 18:40:25.802109 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.802090    2581 scope.go:117] "RemoveContainer" containerID="4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"
Jul 28 18:40:25.802403 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:25.802383    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d\": container with ID starting with 4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d not found: ID does not exist" containerID="4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"
Jul 28 18:40:25.802459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.802413    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d"} err="failed to get container status \"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d\": rpc error: code = NotFound desc = could not find container \"4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d\": container with ID starting with 4165f095a509d75f573d27c6b23ac6dfbf168e86400479f7d9055a5ef350682d not found: ID does not exist"
Jul 28 18:40:25.802459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.802430    2581 scope.go:117] "RemoveContainer" containerID="46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"
Jul 28 18:40:25.810308 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.810282    2581 scope.go:117] "RemoveContainer" containerID="46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"
Jul 28 18:40:25.810552 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:25.810533    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9\": container with ID starting with 46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9 not found: ID does not exist" containerID="46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"
Jul 28 18:40:25.810616 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.810559    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9"} err="failed to get container status \"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9\": rpc error: code = NotFound desc = could not find container \"46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9\": container with ID starting with 46acabef92d33ab62cfa1e073823839d2a34936183403e7851a1fca8f08d8cc9 not found: ID does not exist"
Jul 28 18:40:25.821825 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.821789    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:40:25.827569 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.827546    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-51a15d-5584595454-rwlsc"]
Jul 28 18:40:25.840235 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.840207    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:40:25.847716 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.847689    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-51a15d-58fd864fb-xq4dh"]
Jul 28 18:40:25.863322 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.863295    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:40:25.867712 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:25.867683    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-51a15d-8c454856d-68hcv"]
Jul 28 18:40:26.964351 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:26.964321    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" path="/var/lib/kubelet/pods/2c4eba21-e660-4e90-a19e-2d0dc7f0516e/volumes"
Jul 28 18:40:26.964711 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:26.964660    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" path="/var/lib/kubelet/pods/34e1f043-bdab-43aa-be36-99b57a0c3ce5/volumes"
Jul 28 18:40:26.965013 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:26.964999    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" path="/var/lib/kubelet/pods/a409c864-b3b1-4cf3-a562-3b48ed1ac50c/volumes"
Jul 28 18:40:35.624832 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.624788    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625377    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" containerName="maas-api"
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625398    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" containerName="maas-api"
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625417    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" containerName="payload-processing"
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625426    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" containerName="payload-processing"
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625440    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511c74af-ccc4-4d1c-959c-abe890548598" containerName="payload-processing"
Jul 28 18:40:35.625447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625449    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="511c74af-ccc4-4d1c-959c-abe890548598" containerName="payload-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625461    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" containerName="payload-pre-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625469    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" containerName="payload-pre-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625483    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625491    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625513    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d40c625-f922-4a21-a35f-510366121dbd" containerName="maas-api"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625520    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d40c625-f922-4a21-a35f-510366121dbd" containerName="maas-api"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625600    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a409c864-b3b1-4cf3-a562-3b48ed1ac50c" containerName="maas-api"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625612    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e1f043-bdab-43aa-be36-99b57a0c3ce5" containerName="payload-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625625    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d40c625-f922-4a21-a35f-510366121dbd" containerName="maas-api"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625635    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="511c74af-ccc4-4d1c-959c-abe890548598" containerName="payload-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625645    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db368b1-809e-4d0f-bb6e-51fd10709efb" containerName="payload-pre-processing"
Jul 28 18:40:35.625787 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.625655    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c4eba21-e660-4e90-a19e-2d0dc7f0516e" containerName="payload-pre-processing"
Jul 28 18:40:35.630793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.630768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.633793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.633770    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5523b7-serving-cert\""
Jul 28 18:40:35.641624 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.641599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:40:35.741341 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.741307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hztsh\" (UniqueName: \"kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.741518 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.741422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.741633 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.741610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.843128 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.843092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.843335 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.843147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hztsh\" (UniqueName: \"kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.843335 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.843189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.843335 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:35.843235    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-serving-cert: secret "maas-api-e2e-shared-a-5523b7-serving-cert" not found
Jul 28 18:40:35.843335 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:40:35.843335    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls podName:22a752a7-55f6-4852-8bdc-34ea6a7262a8 nodeName:}" failed. No retries permitted until 2026-07-28 18:40:36.343314873 +0000 UTC m=+2615.913377691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls") pod "maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" (UID: "22a752a7-55f6-4852-8bdc-34ea6a7262a8") : secret "maas-api-e2e-shared-a-5523b7-serving-cert" not found
Jul 28 18:40:35.843857 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.843838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:35.860565 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:35.860537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hztsh\" (UniqueName: \"kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:36.349355 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.349316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:36.351778 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.351752    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") pod \"maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:36.542742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.542708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:36.703285 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.703241    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:40:36.704955 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:40:36.704926    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a752a7_55f6_4852_8bdc_34ea6a7262a8.slice/crio-075ec0e097566aa440549c2a6f60f04be6bdafd8d82f27546af34741854ec4e8 WatchSource:0}: Error finding container 075ec0e097566aa440549c2a6f60f04be6bdafd8d82f27546af34741854ec4e8: Status 404 returned error can't find the container with id 075ec0e097566aa440549c2a6f60f04be6bdafd8d82f27546af34741854ec4e8
Jul 28 18:40:36.705528 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.705504    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:40:36.710885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.710864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.713992 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.713971    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-5523b7-dockercfg-9995t\""
Jul 28 18:40:36.714102 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.713969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-5523b7\""
Jul 28 18:40:36.721684 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.721657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:40:36.730063 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.730037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:40:36.733767 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.733746    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.746640 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.746609    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:40:36.753114 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.753082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.753265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.753171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4hqp\" (UniqueName: \"kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.841232 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.841189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" event={"ID":"22a752a7-55f6-4852-8bdc-34ea6a7262a8","Type":"ContainerStarted","Data":"075ec0e097566aa440549c2a6f60f04be6bdafd8d82f27546af34741854ec4e8"}
Jul 28 18:40:36.853699 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.853666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.853872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.853711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84hg\" (UniqueName: \"kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.853872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.853750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4hqp\" (UniqueName: \"kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.853872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.853806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.854648 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.854623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.863261 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.863229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4hqp\" (UniqueName: \"kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp\") pod \"payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:36.955429 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.955334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.955586 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.955426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84hg\" (UniqueName: \"kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.956905 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.956874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:36.965381 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:36.965343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84hg\" (UniqueName: \"kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg\") pod \"payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:37.022488 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.022448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:37.045695 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.045665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:37.184545 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.184494    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:40:37.189445 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:40:37.189411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19f4ebe_e5d9_455d_a6ff_05b11b5a74a2.slice/crio-8dae941f890dcf238ca331cda0b3f26d666df1ac34008f232a4a5a38ee131bed WatchSource:0}: Error finding container 8dae941f890dcf238ca331cda0b3f26d666df1ac34008f232a4a5a38ee131bed: Status 404 returned error can't find the container with id 8dae941f890dcf238ca331cda0b3f26d666df1ac34008f232a4a5a38ee131bed
Jul 28 18:40:37.214231 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.214203    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:40:37.217820 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:40:37.217789    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cdbc935_fe7f_463a_b89e_cad626bdb712.slice/crio-beea052051afef0b6e81a115f656679edf3d0f234c5da7abf9a9c9e1884bb8af WatchSource:0}: Error finding container beea052051afef0b6e81a115f656679edf3d0f234c5da7abf9a9c9e1884bb8af: Status 404 returned error can't find the container with id beea052051afef0b6e81a115f656679edf3d0f234c5da7abf9a9c9e1884bb8af
Jul 28 18:40:37.847599 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.847561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" event={"ID":"22a752a7-55f6-4852-8bdc-34ea6a7262a8","Type":"ContainerStarted","Data":"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"}
Jul 28 18:40:37.848054 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.847722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:37.848975 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.848951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" event={"ID":"9cdbc935-fe7f-463a-b89e-cad626bdb712","Type":"ContainerStarted","Data":"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"}
Jul 28 18:40:37.849111 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.848996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" event={"ID":"9cdbc935-fe7f-463a-b89e-cad626bdb712","Type":"ContainerStarted","Data":"beea052051afef0b6e81a115f656679edf3d0f234c5da7abf9a9c9e1884bb8af"}
Jul 28 18:40:37.849111 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.849024    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:40:37.850306 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.850263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" event={"ID":"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2","Type":"ContainerStarted","Data":"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"}
Jul 28 18:40:37.850401 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.850323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" event={"ID":"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2","Type":"ContainerStarted","Data":"8dae941f890dcf238ca331cda0b3f26d666df1ac34008f232a4a5a38ee131bed"}
Jul 28 18:40:37.850401 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.850344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:37.866625 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.866551    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" podStartSLOduration=2.412765764 podStartE2EDuration="2.866535002s" totalImagesPullingTime="453.769238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:40:37.865241229 +0000 UTC m=+2617.435304061" watchObservedRunningTime="2026-07-28 18:40:37.866535002 +0000 UTC m=+2617.436597833"
Jul 28 18:40:37.882569 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.882519    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" podStartSLOduration=1.8825011950000001 podStartE2EDuration="1.882501195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:40:37.880902197 +0000 UTC m=+2617.450965029" watchObservedRunningTime="2026-07-28 18:40:37.882501195 +0000 UTC m=+2617.452564025"
Jul 28 18:40:37.903196 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:37.903146    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" podStartSLOduration=1.903128419 podStartE2EDuration="1.903128419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:40:37.90116927 +0000 UTC m=+2617.471232103" watchObservedRunningTime="2026-07-28 18:40:37.903128419 +0000 UTC m=+2617.473191250"
Jul 28 18:40:43.860796 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:43.860757    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:40:48.857696 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:48.857667    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:40:48.858174 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:40:48.857735    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:41:04.950839 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:04.950802    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:41:04.956228 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:04.956204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:04.959606 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:04.959568    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-10fb8f-serving-cert\""
Jul 28 18:41:04.968265 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:04.968227    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:41:05.128071 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.128030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pdg2\" (UniqueName: \"kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.128292 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.128110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.128292 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.128211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.228821 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.228720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.228821 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.228809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.229060 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.228870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pdg2\" (UniqueName: \"kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.229060 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:41:05.228981    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-serving-cert: secret "maas-api-e2e-shared-b-10fb8f-serving-cert" not found
Jul 28 18:41:05.229172 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:41:05.229060    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls podName:531b317f-fb7d-4069-ab65-9edf89b7f526 nodeName:}" failed. No retries permitted until 2026-07-28 18:41:05.729037731 +0000 UTC m=+2645.299100558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls") pod "maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" (UID: "531b317f-fb7d-4069-ab65-9edf89b7f526") : secret "maas-api-e2e-shared-b-10fb8f-serving-cert" not found
Jul 28 18:41:05.229426 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.229407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.244066 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.244035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pdg2\" (UniqueName: \"kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.662547 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.662510    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:41:05.666224 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.666205    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.669689 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.669659    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-10fb8f\""
Jul 28 18:41:05.682220 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.682193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:41:05.696215 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.696186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:41:05.700239 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.700214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.702928 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.702908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-10fb8f-dockercfg-fpbds\""
Jul 28 18:41:05.706860 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.706831    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:41:05.733218 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.733188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.735581 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.735553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") pod \"maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.834601 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.834559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2wv\" (UniqueName: \"kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.834806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.834650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.834806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.834681    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx7rw\" (UniqueName: \"kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.834806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.834720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.869607 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.869573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:05.937119 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.935917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.937119 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.936060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2wv\" (UniqueName: \"kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.937119 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.936154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.937119 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.936191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx7rw\" (UniqueName: \"kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.937738 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.937503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.937738 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.937520    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.945868 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.945837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx7rw\" (UniqueName: \"kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw\") pod \"payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:05.946358 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.946340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2wv\" (UniqueName: \"kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv\") pod \"payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") " pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:05.977687 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:05.977656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:06.013224 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.013182    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:06.024239 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.024209    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:41:06.027059 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:41:06.027014    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531b317f_fb7d_4069_ab65_9edf89b7f526.slice/crio-12a9c68491d65a70afe8a7dbf672ae332f9c4f0eb2ee601fb3505417e31b8d9e WatchSource:0}: Error finding container 12a9c68491d65a70afe8a7dbf672ae332f9c4f0eb2ee601fb3505417e31b8d9e: Status 404 returned error can't find the container with id 12a9c68491d65a70afe8a7dbf672ae332f9c4f0eb2ee601fb3505417e31b8d9e
Jul 28 18:41:06.149581 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.149551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:41:06.150209 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:41:06.150167    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde89c7b7_b626_4801_83b1_90fb941bdbd8.slice/crio-a0ba8df96ccbb56f38e34a99fbd7ad5e328cfbded1c96b7bedc1f12c8bd57642 WatchSource:0}: Error finding container a0ba8df96ccbb56f38e34a99fbd7ad5e328cfbded1c96b7bedc1f12c8bd57642: Status 404 returned error can't find the container with id a0ba8df96ccbb56f38e34a99fbd7ad5e328cfbded1c96b7bedc1f12c8bd57642
Jul 28 18:41:06.178506 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.178480    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:41:06.186604 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:41:06.186569    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb959e62d_cc44_4d16_814c_2db90c91ba54.slice/crio-a2192ae39c247badd32258ae825333c55e9c94e851625f77dcf93301a20cd5b0 WatchSource:0}: Error finding container a2192ae39c247badd32258ae825333c55e9c94e851625f77dcf93301a20cd5b0: Status 404 returned error can't find the container with id a2192ae39c247badd32258ae825333c55e9c94e851625f77dcf93301a20cd5b0
Jul 28 18:41:06.993055 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.993016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" event={"ID":"531b317f-fb7d-4069-ab65-9edf89b7f526","Type":"ContainerStarted","Data":"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"}
Jul 28 18:41:06.993590 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.993079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" event={"ID":"531b317f-fb7d-4069-ab65-9edf89b7f526","Type":"ContainerStarted","Data":"12a9c68491d65a70afe8a7dbf672ae332f9c4f0eb2ee601fb3505417e31b8d9e"}
Jul 28 18:41:06.993590 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.993124    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:06.994729 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.994704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" event={"ID":"b959e62d-cc44-4d16-814c-2db90c91ba54","Type":"ContainerStarted","Data":"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"}
Jul 28 18:41:06.994857 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.994748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" event={"ID":"b959e62d-cc44-4d16-814c-2db90c91ba54","Type":"ContainerStarted","Data":"a2192ae39c247badd32258ae825333c55e9c94e851625f77dcf93301a20cd5b0"}
Jul 28 18:41:06.994857 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.994775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:06.996300 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.996254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" event={"ID":"de89c7b7-b626-4801-83b1-90fb941bdbd8","Type":"ContainerStarted","Data":"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"}
Jul 28 18:41:06.996408 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.996313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" event={"ID":"de89c7b7-b626-4801-83b1-90fb941bdbd8","Type":"ContainerStarted","Data":"a0ba8df96ccbb56f38e34a99fbd7ad5e328cfbded1c96b7bedc1f12c8bd57642"}
Jul 28 18:41:06.996408 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:06.996342    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:41:07.011892 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:07.011845    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" podStartSLOduration=2.638115548 podStartE2EDuration="3.011832055s" totalImagesPullingTime="373.716507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:41:07.010562138 +0000 UTC m=+2646.580624968" watchObservedRunningTime="2026-07-28 18:41:07.011832055 +0000 UTC m=+2646.581894885"
Jul 28 18:41:07.029038 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:07.028987    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" podStartSLOduration=2.02897338 podStartE2EDuration="2.02897338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:41:07.026771385 +0000 UTC m=+2646.596834217" watchObservedRunningTime="2026-07-28 18:41:07.02897338 +0000 UTC m=+2646.599036211"
Jul 28 18:41:07.042773 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:07.042722    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" podStartSLOduration=2.042707114 podStartE2EDuration="2.042707114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:41:07.041807364 +0000 UTC m=+2646.611870197" watchObservedRunningTime="2026-07-28 18:41:07.042707114 +0000 UTC m=+2646.612769944"
Jul 28 18:41:13.007769 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:13.007738    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:41:18.003722 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:18.003687    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:41:18.004216 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:41:18.004079    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:51:20.480242 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.480208    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:51:20.483952 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.483932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.486326 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.486299    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-473596-serving-cert\""
Jul 28 18:51:20.501925 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.501896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:51:20.614860 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.614819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.615078 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.614878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.615078 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.614946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgmmg\" (UniqueName: \"kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.715900 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.715856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.716105 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.715947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgmmg\" (UniqueName: \"kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.716105 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.716058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.716226 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:51:20.716203    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-serving-cert: secret "maas-api-e2e-ipp-a-473596-serving-cert" not found
Jul 28 18:51:20.716330 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:51:20.716317    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls podName:570f8320-82f0-4841-b6f8-4e3ef5d9cdce nodeName:}" failed. No retries permitted until 2026-07-28 18:51:21.216263538 +0000 UTC m=+3260.786326367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls") pod "maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" (UID: "570f8320-82f0-4841-b6f8-4e3ef5d9cdce") : secret "maas-api-e2e-ipp-a-473596-serving-cert" not found
Jul 28 18:51:20.716636 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.716616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:20.728556 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:20.728528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgmmg\" (UniqueName: \"kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:21.221320 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.221248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:21.223618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.223596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") pod \"maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:21.394523 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.394488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:21.398806 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.398783    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:51:21.402212 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.402195    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.406868 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.406164    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-473596\""
Jul 28 18:51:21.406868 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.406471    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-473596-dockercfg-tljvl\""
Jul 28 18:51:21.416154 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.416121    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:51:21.438525 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.438496    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:51:21.452725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.452693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.469464 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.469418    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:51:21.526515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.526020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.526515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.526084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.526515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.526140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pps\" (UniqueName: \"kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.526515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.526263    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5j5s\" (UniqueName: \"kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.580928 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.580900    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:51:21.583376 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:51:21.583340    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570f8320_82f0_4841_b6f8_4e3ef5d9cdce.slice/crio-e04b18685f6d79edc27ea2825db1974d2c8aaecf06a4fd527e8286cb19efdff9 WatchSource:0}: Error finding container e04b18685f6d79edc27ea2825db1974d2c8aaecf06a4fd527e8286cb19efdff9: Status 404 returned error can't find the container with id e04b18685f6d79edc27ea2825db1974d2c8aaecf06a4fd527e8286cb19efdff9
Jul 28 18:51:21.584712 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.584692    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 18:51:21.627244 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.627209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5j5s\" (UniqueName: \"kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.627441 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.627342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.627441 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.627374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.627441 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.627408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pps\" (UniqueName: \"kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.628107 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.628085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.628152 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.628097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.636618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.636591    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5j5s\" (UniqueName: \"kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s\") pod \"payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.636618 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.636615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pps\" (UniqueName: \"kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps\") pod \"payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.745357 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.745245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:21.775590 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.775558    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:21.907249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.907218    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:51:21.911514 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:51:21.911481    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee71262_8970_4f78_85c9_0a389ec20fd6.slice/crio-899c2cc02c1d1d326b2b3395789e240dbf7a4c9810171b5b3ee9b8e763c91187 WatchSource:0}: Error finding container 899c2cc02c1d1d326b2b3395789e240dbf7a4c9810171b5b3ee9b8e763c91187: Status 404 returned error can't find the container with id 899c2cc02c1d1d326b2b3395789e240dbf7a4c9810171b5b3ee9b8e763c91187
Jul 28 18:51:21.917161 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:21.917137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" event={"ID":"570f8320-82f0-4841-b6f8-4e3ef5d9cdce","Type":"ContainerStarted","Data":"e04b18685f6d79edc27ea2825db1974d2c8aaecf06a4fd527e8286cb19efdff9"}
Jul 28 18:51:22.144475 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.144381    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:51:22.158254 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:51:22.158227    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683f477c_efdb_44f7_9723_5c779ceba872.slice/crio-45f509d4962a276cc05b0e123ea8b29822287e062fe388935874bf7b219618dc WatchSource:0}: Error finding container 45f509d4962a276cc05b0e123ea8b29822287e062fe388935874bf7b219618dc: Status 404 returned error can't find the container with id 45f509d4962a276cc05b0e123ea8b29822287e062fe388935874bf7b219618dc
Jul 28 18:51:22.923634 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.923598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" event={"ID":"570f8320-82f0-4841-b6f8-4e3ef5d9cdce","Type":"ContainerStarted","Data":"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"}
Jul 28 18:51:22.924099 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.923690    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:22.925321 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.925265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" event={"ID":"683f477c-efdb-44f7-9723-5c779ceba872","Type":"ContainerStarted","Data":"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"}
Jul 28 18:51:22.925459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.925341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" event={"ID":"683f477c-efdb-44f7-9723-5c779ceba872","Type":"ContainerStarted","Data":"45f509d4962a276cc05b0e123ea8b29822287e062fe388935874bf7b219618dc"}
Jul 28 18:51:22.925459 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.925372    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:22.926759 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.926735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" event={"ID":"8ee71262-8970-4f78-85c9-0a389ec20fd6","Type":"ContainerStarted","Data":"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"}
Jul 28 18:51:22.926872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.926774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" event={"ID":"8ee71262-8970-4f78-85c9-0a389ec20fd6","Type":"ContainerStarted","Data":"899c2cc02c1d1d326b2b3395789e240dbf7a4c9810171b5b3ee9b8e763c91187"}
Jul 28 18:51:22.926872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.926807    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:22.942551 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.942497    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" podStartSLOduration=2.424020963 podStartE2EDuration="2.942478584s" totalImagesPullingTime="518.457621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:22.93972882 +0000 UTC m=+3262.509791684" watchObservedRunningTime="2026-07-28 18:51:22.942478584 +0000 UTC m=+3262.512541417"
Jul 28 18:51:22.956875 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.956824    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" podStartSLOduration=1.956809524 podStartE2EDuration="1.956809524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:22.954871747 +0000 UTC m=+3262.524934579" watchObservedRunningTime="2026-07-28 18:51:22.956809524 +0000 UTC m=+3262.526872356"
Jul 28 18:51:22.972925 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:22.972869    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" podStartSLOduration=1.9728532890000001 podStartE2EDuration="1.972853289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:22.970557566 +0000 UTC m=+3262.540620398" watchObservedRunningTime="2026-07-28 18:51:22.972853289 +0000 UTC m=+3262.542916121"
Jul 28 18:51:28.937012 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:28.936983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:51:33.934161 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:33.934122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:51:33.934585 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:33.934199    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:51:50.211415 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.211378    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:51:50.214627 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.214610    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.219735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.219715    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5b063e-dockercfg-8kqgl\""
Jul 28 18:51:50.219903 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.219885    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5b063e\""
Jul 28 18:51:50.229560 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.229524    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:51:50.231393 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.231368    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:51:50.234750 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.234723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.247472 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.247444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:51:50.303704 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.303664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqxmz\" (UniqueName: \"kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.303912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.303756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.303912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.303825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.303912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.303859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2wtw\" (UniqueName: \"kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.405231 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.405196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2wtw\" (UniqueName: \"kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.405432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.405248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqxmz\" (UniqueName: \"kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.405432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.405320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.405432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.405368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.406301 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.406260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.406351 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.406259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.413824 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.413788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2wtw\" (UniqueName: \"kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw\") pod \"payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.413959 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.413846    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqxmz\" (UniqueName: \"kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz\") pod \"payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.526216 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.526120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:50.548324 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.548297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:50.688243 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.688209    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:51:50.689944 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:51:50.689913    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc740f049_8c04_44e9_942d_c237649ea780.slice/crio-67d7cd830955b8830246e1950bc11fbd64a28e41914d34ee7c3fe227b42af4bd WatchSource:0}: Error finding container 67d7cd830955b8830246e1950bc11fbd64a28e41914d34ee7c3fe227b42af4bd: Status 404 returned error can't find the container with id 67d7cd830955b8830246e1950bc11fbd64a28e41914d34ee7c3fe227b42af4bd
Jul 28 18:51:50.923825 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:50.923794    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:51:50.929636 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:51:50.929584    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7c1ad2_ea0a_41be_ab44_c787f517eaa0.slice/crio-029f71fefb9c0251f2606b5189cae7506ad5839e8f7f83ea6f343b79c8e03eb9 WatchSource:0}: Error finding container 029f71fefb9c0251f2606b5189cae7506ad5839e8f7f83ea6f343b79c8e03eb9: Status 404 returned error can't find the container with id 029f71fefb9c0251f2606b5189cae7506ad5839e8f7f83ea6f343b79c8e03eb9
Jul 28 18:51:51.075555 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.075519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" event={"ID":"c740f049-8c04-44e9-942d-c237649ea780","Type":"ContainerStarted","Data":"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"}
Jul 28 18:51:51.075761 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.075575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" event={"ID":"c740f049-8c04-44e9-942d-c237649ea780","Type":"ContainerStarted","Data":"67d7cd830955b8830246e1950bc11fbd64a28e41914d34ee7c3fe227b42af4bd"}
Jul 28 18:51:51.075761 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.075629    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:51:51.077439 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.077409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" event={"ID":"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0","Type":"ContainerStarted","Data":"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"}
Jul 28 18:51:51.077591 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.077459    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" event={"ID":"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0","Type":"ContainerStarted","Data":"029f71fefb9c0251f2606b5189cae7506ad5839e8f7f83ea6f343b79c8e03eb9"}
Jul 28 18:51:51.077591 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.077487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:51:51.093255 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.093202    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" podStartSLOduration=1.093185942 podStartE2EDuration="1.093185942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:51.091062865 +0000 UTC m=+3290.661125699" watchObservedRunningTime="2026-07-28 18:51:51.093185942 +0000 UTC m=+3290.663248773"
Jul 28 18:51:51.110625 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:51:51.110564    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" podStartSLOduration=1.110540757 podStartE2EDuration="1.110540757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:51:51.109925216 +0000 UTC m=+3290.679988049" watchObservedRunningTime="2026-07-28 18:51:51.110540757 +0000 UTC m=+3290.680603590"
Jul 28 18:52:02.084817 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:52:02.084788    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:52:02.085334 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:52:02.084861    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:52:21.401896 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:52:21.401862    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd_fe7c1ad2-ea0a-41be-ab44-c787f517eaa0/payload-processing/0.log"
Jul 28 18:53:16.473573 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:16.473540    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4_683f477c-efdb-44f7-9723-5c779ceba872/payload-processing/0.log"
Jul 28 18:53:16.726250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:16.726172    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd_fe7c1ad2-ea0a-41be-ab44-c787f517eaa0/payload-processing/0.log"
Jul 28 18:53:21.640959 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.640916    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:21.644884 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.644860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.647513 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.647492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-94ebbf-dockercfg-kc7kf\""
Jul 28 18:53:21.647622 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.647553    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-94ebbf\""
Jul 28 18:53:21.654728 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.654702    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:21.808330 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.808288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.808515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.808358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm5b8\" (UniqueName: \"kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.909814 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.909712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.909814 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.909774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm5b8\" (UniqueName: \"kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.910543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.910515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.918083 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.918055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm5b8\" (UniqueName: \"kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8\") pod \"payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:21.956725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:21.956690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:22.106550 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:22.106514    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:22.109437 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:53:22.109403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f70b1ec_9d3e_4996_905b_68f212655c2b.slice/crio-e0366fd081b4eaa1ed595074e7937a1e8bdf77a07a6be3514043d95cd4ac7642 WatchSource:0}: Error finding container e0366fd081b4eaa1ed595074e7937a1e8bdf77a07a6be3514043d95cd4ac7642: Status 404 returned error can't find the container with id e0366fd081b4eaa1ed595074e7937a1e8bdf77a07a6be3514043d95cd4ac7642
Jul 28 18:53:22.544885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:22.544775    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" event={"ID":"3f70b1ec-9d3e-4996-905b-68f212655c2b","Type":"ContainerStarted","Data":"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"}
Jul 28 18:53:22.544885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:22.544841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" event={"ID":"3f70b1ec-9d3e-4996-905b-68f212655c2b","Type":"ContainerStarted","Data":"e0366fd081b4eaa1ed595074e7937a1e8bdf77a07a6be3514043d95cd4ac7642"}
Jul 28 18:53:22.544885 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:22.544864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:22.560498 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:22.560446    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" podStartSLOduration=1.560429138 podStartE2EDuration="1.560429138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:53:22.559023839 +0000 UTC m=+3382.129086671" watchObservedRunningTime="2026-07-28 18:53:22.560429138 +0000 UTC m=+3382.130491969"
Jul 28 18:53:33.552789 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:33.552756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:51.600838 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.600798    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:51.601680 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.601073    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" podUID="3f70b1ec-9d3e-4996-905b-68f212655c2b" containerName="payload-pre-processing" containerID="cri-o://d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203" gracePeriod=30
Jul 28 18:53:51.875911 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.875883    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:51.998550 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.998514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm5b8\" (UniqueName: \"kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8\") pod \"3f70b1ec-9d3e-4996-905b-68f212655c2b\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") "
Jul 28 18:53:51.998725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.998573    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume\") pod \"3f70b1ec-9d3e-4996-905b-68f212655c2b\" (UID: \"3f70b1ec-9d3e-4996-905b-68f212655c2b\") "
Jul 28 18:53:51.998982 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:51.998953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f70b1ec-9d3e-4996-905b-68f212655c2b" (UID: "3f70b1ec-9d3e-4996-905b-68f212655c2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:53:52.000665 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.000642    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8" (OuterVolumeSpecName: "kube-api-access-mm5b8") pod "3f70b1ec-9d3e-4996-905b-68f212655c2b" (UID: "3f70b1ec-9d3e-4996-905b-68f212655c2b"). InnerVolumeSpecName "kube-api-access-mm5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:53:52.100457 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.100418    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm5b8\" (UniqueName: \"kubernetes.io/projected/3f70b1ec-9d3e-4996-905b-68f212655c2b-kube-api-access-mm5b8\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:53:52.100457 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.100450    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f70b1ec-9d3e-4996-905b-68f212655c2b-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:53:52.707722 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.707681    2581 generic.go:356] "Generic (PLEG): container finished" podID="3f70b1ec-9d3e-4996-905b-68f212655c2b" containerID="d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203" exitCode=0
Jul 28 18:53:52.708142 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.707737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"
Jul 28 18:53:52.708142 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.707768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" event={"ID":"3f70b1ec-9d3e-4996-905b-68f212655c2b","Type":"ContainerDied","Data":"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"}
Jul 28 18:53:52.708142 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.707844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx" event={"ID":"3f70b1ec-9d3e-4996-905b-68f212655c2b","Type":"ContainerDied","Data":"e0366fd081b4eaa1ed595074e7937a1e8bdf77a07a6be3514043d95cd4ac7642"}
Jul 28 18:53:52.708142 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.707867    2581 scope.go:117] "RemoveContainer" containerID="d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"
Jul 28 18:53:52.717817 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.717801    2581 scope.go:117] "RemoveContainer" containerID="d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"
Jul 28 18:53:52.718082 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:53:52.718062    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203\": container with ID starting with d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203 not found: ID does not exist" containerID="d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"
Jul 28 18:53:52.718146 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.718091    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203"} err="failed to get container status \"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203\": rpc error: code = NotFound desc = could not find container \"d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203\": container with ID starting with d2caf22483fc367bdf8e8903ef257cf8bc45af4bc5eb39f5b449cb8b82697203 not found: ID does not exist"
Jul 28 18:53:52.731452 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.731424    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:52.736543 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.736516    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-94ebbf-6df875bf7b-4cvwx"]
Jul 28 18:53:52.965819 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:53:52.965703    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f70b1ec-9d3e-4996-905b-68f212655c2b" path="/var/lib/kubelet/pods/3f70b1ec-9d3e-4996-905b-68f212655c2b/volumes"
Jul 28 18:54:18.129560 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.129523    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:54:18.130039 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.129756    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" podUID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" containerName="maas-api" containerID="cri-o://7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4" gracePeriod=30
Jul 28 18:54:18.187725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.187693    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:54:18.187949 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.187923    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" podUID="683f477c-efdb-44f7-9723-5c779ceba872" containerName="payload-processing" containerID="cri-o://864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84" gracePeriod=30
Jul 28 18:54:18.193993 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.193866    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:54:18.194156 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.194085    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" podUID="8ee71262-8970-4f78-85c9-0a389ec20fd6" containerName="payload-pre-processing" containerID="cri-o://b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59" gracePeriod=30
Jul 28 18:54:18.508552 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.508523    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:54:18.512227 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.512203    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:54:18.519605 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.519582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:54:18.560664 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560634    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5j5s\" (UniqueName: \"kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s\") pod \"683f477c-efdb-44f7-9723-5c779ceba872\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") "
Jul 28 18:54:18.560852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560695    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgmmg\" (UniqueName: \"kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg\") pod \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") "
Jul 28 18:54:18.560852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") pod \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") "
Jul 28 18:54:18.560852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume\") pod \"8ee71262-8970-4f78-85c9-0a389ec20fd6\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") "
Jul 28 18:54:18.560852 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6pps\" (UniqueName: \"kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps\") pod \"8ee71262-8970-4f78-85c9-0a389ec20fd6\" (UID: \"8ee71262-8970-4f78-85c9-0a389ec20fd6\") "
Jul 28 18:54:18.561087 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560883    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume\") pod \"683f477c-efdb-44f7-9723-5c779ceba872\" (UID: \"683f477c-efdb-44f7-9723-5c779ceba872\") "
Jul 28 18:54:18.561087 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.560917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca\") pod \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\" (UID: \"570f8320-82f0-4841-b6f8-4e3ef5d9cdce\") "
Jul 28 18:54:18.561392 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.561361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ee71262-8970-4f78-85c9-0a389ec20fd6" (UID: "8ee71262-8970-4f78-85c9-0a389ec20fd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:54:18.561488 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.561466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "683f477c-efdb-44f7-9723-5c779ceba872" (UID: "683f477c-efdb-44f7-9723-5c779ceba872"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:54:18.561682 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.561654    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "570f8320-82f0-4841-b6f8-4e3ef5d9cdce" (UID: "570f8320-82f0-4841-b6f8-4e3ef5d9cdce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:54:18.562950 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.562921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s" (OuterVolumeSpecName: "kube-api-access-w5j5s") pod "683f477c-efdb-44f7-9723-5c779ceba872" (UID: "683f477c-efdb-44f7-9723-5c779ceba872"). InnerVolumeSpecName "kube-api-access-w5j5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:18.563591 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.563562    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg" (OuterVolumeSpecName: "kube-api-access-qgmmg") pod "570f8320-82f0-4841-b6f8-4e3ef5d9cdce" (UID: "570f8320-82f0-4841-b6f8-4e3ef5d9cdce"). InnerVolumeSpecName "kube-api-access-qgmmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:18.563691 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.563656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps" (OuterVolumeSpecName: "kube-api-access-m6pps") pod "8ee71262-8970-4f78-85c9-0a389ec20fd6" (UID: "8ee71262-8970-4f78-85c9-0a389ec20fd6"). InnerVolumeSpecName "kube-api-access-m6pps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:54:18.563740 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.563720    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "570f8320-82f0-4841-b6f8-4e3ef5d9cdce" (UID: "570f8320-82f0-4841-b6f8-4e3ef5d9cdce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:54:18.662742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662704    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6pps\" (UniqueName: \"kubernetes.io/projected/8ee71262-8970-4f78-85c9-0a389ec20fd6-kube-api-access-m6pps\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662742 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662738    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/683f477c-efdb-44f7-9723-5c779ceba872-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662751    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662766    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5j5s\" (UniqueName: \"kubernetes.io/projected/683f477c-efdb-44f7-9723-5c779ceba872-kube-api-access-w5j5s\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662778    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgmmg\" (UniqueName: \"kubernetes.io/projected/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-kube-api-access-qgmmg\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662792    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570f8320-82f0-4841-b6f8-4e3ef5d9cdce-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.662962 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.662803    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee71262-8970-4f78-85c9-0a389ec20fd6-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:54:18.844542 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.844507    2581 generic.go:356] "Generic (PLEG): container finished" podID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" containerID="7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4" exitCode=0
Jul 28 18:54:18.844735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.844552    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"
Jul 28 18:54:18.844735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.844588    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" event={"ID":"570f8320-82f0-4841-b6f8-4e3ef5d9cdce","Type":"ContainerDied","Data":"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"}
Jul 28 18:54:18.844735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.844642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk" event={"ID":"570f8320-82f0-4841-b6f8-4e3ef5d9cdce","Type":"ContainerDied","Data":"e04b18685f6d79edc27ea2825db1974d2c8aaecf06a4fd527e8286cb19efdff9"}
Jul 28 18:54:18.844735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.844670    2581 scope.go:117] "RemoveContainer" containerID="7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"
Jul 28 18:54:18.845891 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.845870    2581 generic.go:356] "Generic (PLEG): container finished" podID="683f477c-efdb-44f7-9723-5c779ceba872" containerID="864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84" exitCode=0
Jul 28 18:54:18.845991 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.845907    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"
Jul 28 18:54:18.845991 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.845966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" event={"ID":"683f477c-efdb-44f7-9723-5c779ceba872","Type":"ContainerDied","Data":"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"}
Jul 28 18:54:18.846102 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.846018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4" event={"ID":"683f477c-efdb-44f7-9723-5c779ceba872","Type":"ContainerDied","Data":"45f509d4962a276cc05b0e123ea8b29822287e062fe388935874bf7b219618dc"}
Jul 28 18:54:18.847576 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.847550    2581 generic.go:356] "Generic (PLEG): container finished" podID="8ee71262-8970-4f78-85c9-0a389ec20fd6" containerID="b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59" exitCode=0
Jul 28 18:54:18.847715 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.847590    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"
Jul 28 18:54:18.847715 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.847632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" event={"ID":"8ee71262-8970-4f78-85c9-0a389ec20fd6","Type":"ContainerDied","Data":"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"}
Jul 28 18:54:18.847715 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.847679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw" event={"ID":"8ee71262-8970-4f78-85c9-0a389ec20fd6","Type":"ContainerDied","Data":"899c2cc02c1d1d326b2b3395789e240dbf7a4c9810171b5b3ee9b8e763c91187"}
Jul 28 18:54:18.861666 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.861637    2581 scope.go:117] "RemoveContainer" containerID="7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"
Jul 28 18:54:18.862208 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:54:18.862180    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4\": container with ID starting with 7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4 not found: ID does not exist" containerID="7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"
Jul 28 18:54:18.862352 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.862311    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4"} err="failed to get container status \"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4\": rpc error: code = NotFound desc = could not find container \"7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4\": container with ID starting with 7f064f4188efc710dbc5fb2575d8157b6c18c60e1feaaac4f9b0469476f0bcc4 not found: ID does not exist"
Jul 28 18:54:18.862443 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.862349    2581 scope.go:117] "RemoveContainer" containerID="864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"
Jul 28 18:54:18.872001 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.871983    2581 scope.go:117] "RemoveContainer" containerID="864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"
Jul 28 18:54:18.872261 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:54:18.872239    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84\": container with ID starting with 864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84 not found: ID does not exist" containerID="864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"
Jul 28 18:54:18.872376 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.872281    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84"} err="failed to get container status \"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84\": rpc error: code = NotFound desc = could not find container \"864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84\": container with ID starting with 864400898e1c2d6e6cc4c39c0e0b1b3b32a538bdb99407efcfa0363867c7de84 not found: ID does not exist"
Jul 28 18:54:18.872376 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.872301    2581 scope.go:117] "RemoveContainer" containerID="b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"
Jul 28 18:54:18.880967 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.880941    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:54:18.881262 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.881230    2581 scope.go:117] "RemoveContainer" containerID="b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"
Jul 28 18:54:18.881579 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:54:18.881560    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59\": container with ID starting with b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59 not found: ID does not exist" containerID="b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"
Jul 28 18:54:18.881640 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.881587    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59"} err="failed to get container status \"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59\": rpc error: code = NotFound desc = could not find container \"b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59\": container with ID starting with b1fc24156f0dcaa0a31bdeb08c547ca6d2639da4ec81ff95917b1991debc4d59 not found: ID does not exist"
Jul 28 18:54:18.882974 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.882950    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-473596-546fdb9cc9-76tsk"]
Jul 28 18:54:18.895556 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.895531    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:54:18.899985 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.899959    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-473596-5d6bfc4cfd-dvsv4"]
Jul 28 18:54:18.911775 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.911748    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:54:18.915548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.915520    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-473596-5d9665bdd-9ktcw"]
Jul 28 18:54:18.964363 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.964333    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" path="/var/lib/kubelet/pods/570f8320-82f0-4841-b6f8-4e3ef5d9cdce/volumes"
Jul 28 18:54:18.964725 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.964713    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683f477c-efdb-44f7-9723-5c779ceba872" path="/var/lib/kubelet/pods/683f477c-efdb-44f7-9723-5c779ceba872/volumes"
Jul 28 18:54:18.965033 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:54:18.965021    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee71262-8970-4f78-85c9-0a389ec20fd6" path="/var/lib/kubelet/pods/8ee71262-8970-4f78-85c9-0a389ec20fd6/volumes"
Jul 28 18:55:18.067575 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.067526    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:55:18.068157 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.067827    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" podUID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" containerName="payload-processing" containerID="cri-o://be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788" gracePeriod=30
Jul 28 18:55:18.076170 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.076139    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:55:18.076446 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.076395    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" podUID="c740f049-8c04-44e9-942d-c237649ea780" containerName="payload-pre-processing" containerID="cri-o://5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa" gracePeriod=30
Jul 28 18:55:18.333460 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.333427    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:55:18.371794 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.371769    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:55:18.447583 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.447548    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqxmz\" (UniqueName: \"kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz\") pod \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") "
Jul 28 18:55:18.447764 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.447648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume\") pod \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\" (UID: \"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0\") "
Jul 28 18:55:18.447979 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.447959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" (UID: "fe7c1ad2-ea0a-41be-ab44-c787f517eaa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:18.449773 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.449752    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz" (OuterVolumeSpecName: "kube-api-access-wqxmz") pod "fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" (UID: "fe7c1ad2-ea0a-41be-ab44-c787f517eaa0"). InnerVolumeSpecName "kube-api-access-wqxmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:18.549252 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.549220    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume\") pod \"c740f049-8c04-44e9-942d-c237649ea780\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") "
Jul 28 18:55:18.549447 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.549327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2wtw\" (UniqueName: \"kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw\") pod \"c740f049-8c04-44e9-942d-c237649ea780\" (UID: \"c740f049-8c04-44e9-942d-c237649ea780\") "
Jul 28 18:55:18.549672 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.549646    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c740f049-8c04-44e9-942d-c237649ea780" (UID: "c740f049-8c04-44e9-942d-c237649ea780"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:18.549744 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.549661    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:18.549744 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.549700    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqxmz\" (UniqueName: \"kubernetes.io/projected/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0-kube-api-access-wqxmz\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:18.551358 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.551332    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw" (OuterVolumeSpecName: "kube-api-access-j2wtw") pod "c740f049-8c04-44e9-942d-c237649ea780" (UID: "c740f049-8c04-44e9-942d-c237649ea780"). InnerVolumeSpecName "kube-api-access-j2wtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:18.651064 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.650964    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c740f049-8c04-44e9-942d-c237649ea780-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:18.651064 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:18.651000    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2wtw\" (UniqueName: \"kubernetes.io/projected/c740f049-8c04-44e9-942d-c237649ea780-kube-api-access-j2wtw\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:19.151793 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.151754    2581 generic.go:356] "Generic (PLEG): container finished" podID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" containerID="be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788" exitCode=0
Jul 28 18:55:19.152250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.151809    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"
Jul 28 18:55:19.152250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.151819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" event={"ID":"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0","Type":"ContainerDied","Data":"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"}
Jul 28 18:55:19.152250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.151867    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd" event={"ID":"fe7c1ad2-ea0a-41be-ab44-c787f517eaa0","Type":"ContainerDied","Data":"029f71fefb9c0251f2606b5189cae7506ad5839e8f7f83ea6f343b79c8e03eb9"}
Jul 28 18:55:19.152250 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.151889    2581 scope.go:117] "RemoveContainer" containerID="be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"
Jul 28 18:55:19.153224 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.153204    2581 generic.go:356] "Generic (PLEG): container finished" podID="c740f049-8c04-44e9-942d-c237649ea780" containerID="5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa" exitCode=0
Jul 28 18:55:19.153299 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.153241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" event={"ID":"c740f049-8c04-44e9-942d-c237649ea780","Type":"ContainerDied","Data":"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"}
Jul 28 18:55:19.153299 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.153262    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"
Jul 28 18:55:19.153381 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.153299    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth" event={"ID":"c740f049-8c04-44e9-942d-c237649ea780","Type":"ContainerDied","Data":"67d7cd830955b8830246e1950bc11fbd64a28e41914d34ee7c3fe227b42af4bd"}
Jul 28 18:55:19.161340 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.161323    2581 scope.go:117] "RemoveContainer" containerID="be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"
Jul 28 18:55:19.161607 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:19.161589    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788\": container with ID starting with be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788 not found: ID does not exist" containerID="be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"
Jul 28 18:55:19.161693 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.161621    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788"} err="failed to get container status \"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788\": rpc error: code = NotFound desc = could not find container \"be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788\": container with ID starting with be05d84918fc404973ce6de56c0addcd8b6aec1ebe9ed66ecad3e5efc81e9788 not found: ID does not exist"
Jul 28 18:55:19.161693 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.161647    2581 scope.go:117] "RemoveContainer" containerID="5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"
Jul 28 18:55:19.179016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.173688    2581 scope.go:117] "RemoveContainer" containerID="5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"
Jul 28 18:55:19.179016 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:19.174224    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa\": container with ID starting with 5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa not found: ID does not exist" containerID="5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"
Jul 28 18:55:19.179016 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.174263    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa"} err="failed to get container status \"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa\": rpc error: code = NotFound desc = could not find container \"5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa\": container with ID starting with 5e473a4cd5b4c8a612084517a67c3378d51c25564f740531208a3c89225deeaa not found: ID does not exist"
Jul 28 18:55:19.180364 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.180339    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:55:19.182170 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.182149    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5b063e-dbf87c788-hqqth"]
Jul 28 18:55:19.192083 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.192056    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:55:19.196125 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:19.196102    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5b063e-5c969cb7b6-w2ndd"]
Jul 28 18:55:20.966144 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:20.966107    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c740f049-8c04-44e9-942d-c237649ea780" path="/var/lib/kubelet/pods/c740f049-8c04-44e9-942d-c237649ea780/volumes"
Jul 28 18:55:20.966548 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:20.966523    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" path="/var/lib/kubelet/pods/fe7c1ad2-ea0a-41be-ab44-c787f517eaa0/volumes"
Jul 28 18:55:35.830762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.830675    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:55:35.831352 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.830927    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" podUID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" containerName="maas-api" containerID="cri-o://d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104" gracePeriod=30
Jul 28 18:55:35.889749 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.889706    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:55:35.889996 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.889917    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" podUID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" containerName="payload-processing" containerID="cri-o://f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd" gracePeriod=30
Jul 28 18:55:35.898293 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.898243    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:55:35.898521 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:35.898485    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" podUID="9cdbc935-fe7f-463a-b89e-cad626bdb712" containerName="payload-pre-processing" containerID="cri-o://bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0" gracePeriod=30
Jul 28 18:55:36.202878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.202816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:55:36.206452 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.206425    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:55:36.214710 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.214688    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:55:36.238249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.238103    2581 generic.go:356] "Generic (PLEG): container finished" podID="9cdbc935-fe7f-463a-b89e-cad626bdb712" containerID="bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0" exitCode=0
Jul 28 18:55:36.238249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.238156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" event={"ID":"9cdbc935-fe7f-463a-b89e-cad626bdb712","Type":"ContainerDied","Data":"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"}
Jul 28 18:55:36.238249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.238242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk" event={"ID":"9cdbc935-fe7f-463a-b89e-cad626bdb712","Type":"ContainerDied","Data":"beea052051afef0b6e81a115f656679edf3d0f234c5da7abf9a9c9e1884bb8af"}
Jul 28 18:55:36.238249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.238179    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"
Jul 28 18:55:36.238625 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.238301    2581 scope.go:117] "RemoveContainer" containerID="bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"
Jul 28 18:55:36.241070 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.240945    2581 generic.go:356] "Generic (PLEG): container finished" podID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" containerID="f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd" exitCode=0
Jul 28 18:55:36.241070 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.241044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"
Jul 28 18:55:36.241290 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.241104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" event={"ID":"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2","Type":"ContainerDied","Data":"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"}
Jul 28 18:55:36.241290 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.241147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b" event={"ID":"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2","Type":"ContainerDied","Data":"8dae941f890dcf238ca331cda0b3f26d666df1ac34008f232a4a5a38ee131bed"}
Jul 28 18:55:36.243188 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.243154    2581 generic.go:356] "Generic (PLEG): container finished" podID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" containerID="d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104" exitCode=0
Jul 28 18:55:36.243188 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.243184    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"
Jul 28 18:55:36.243376 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.243245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" event={"ID":"22a752a7-55f6-4852-8bdc-34ea6a7262a8","Type":"ContainerDied","Data":"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"}
Jul 28 18:55:36.243376 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.243306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl" event={"ID":"22a752a7-55f6-4852-8bdc-34ea6a7262a8","Type":"ContainerDied","Data":"075ec0e097566aa440549c2a6f60f04be6bdafd8d82f27546af34741854ec4e8"}
Jul 28 18:55:36.249077 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.249060    2581 scope.go:117] "RemoveContainer" containerID="bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"
Jul 28 18:55:36.249380 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:36.249359    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0\": container with ID starting with bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0 not found: ID does not exist" containerID="bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"
Jul 28 18:55:36.249470 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.249392    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0"} err="failed to get container status \"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0\": rpc error: code = NotFound desc = could not find container \"bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0\": container with ID starting with bab88b492433c40f34386488a97aa97135824cd318bb23699a0535f169ce73d0 not found: ID does not exist"
Jul 28 18:55:36.249470 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.249415    2581 scope.go:117] "RemoveContainer" containerID="f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"
Jul 28 18:55:36.260001 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.259973    2581 scope.go:117] "RemoveContainer" containerID="f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"
Jul 28 18:55:36.260356 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:36.260333    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd\": container with ID starting with f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd not found: ID does not exist" containerID="f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"
Jul 28 18:55:36.260465 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.260366    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd"} err="failed to get container status \"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd\": rpc error: code = NotFound desc = could not find container \"f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd\": container with ID starting with f3c86a893695fe3bc0a3042e29c488921d7713e88a84ca5a90fd0161ea9a55cd not found: ID does not exist"
Jul 28 18:55:36.260465 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.260389    2581 scope.go:117] "RemoveContainer" containerID="d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"
Jul 28 18:55:36.268902 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.268873    2581 scope.go:117] "RemoveContainer" containerID="d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"
Jul 28 18:55:36.269194 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:36.269170    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104\": container with ID starting with d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104 not found: ID does not exist" containerID="d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"
Jul 28 18:55:36.269325 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.269202    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104"} err="failed to get container status \"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104\": rpc error: code = NotFound desc = could not find container \"d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104\": container with ID starting with d79008e48c7ca8e8bea7b0d8d54931ba4a73d3176d1ca5ddc124dea57063a104 not found: ID does not exist"
Jul 28 18:55:36.326907 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.326868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4hqp\" (UniqueName: \"kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp\") pod \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") "
Jul 28 18:55:36.327084 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.326938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b84hg\" (UniqueName: \"kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg\") pod \"9cdbc935-fe7f-463a-b89e-cad626bdb712\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") "
Jul 28 18:55:36.327084 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.326969    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") pod \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") "
Jul 28 18:55:36.327084 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hztsh\" (UniqueName: \"kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh\") pod \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") "
Jul 28 18:55:36.327301 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume\") pod \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\" (UID: \"f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2\") "
Jul 28 18:55:36.327301 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume\") pod \"9cdbc935-fe7f-463a-b89e-cad626bdb712\" (UID: \"9cdbc935-fe7f-463a-b89e-cad626bdb712\") "
Jul 28 18:55:36.327301 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca\") pod \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\" (UID: \"22a752a7-55f6-4852-8bdc-34ea6a7262a8\") "
Jul 28 18:55:36.327643 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327611    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" (UID: "f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:36.327735 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327632    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cdbc935-fe7f-463a-b89e-cad626bdb712" (UID: "9cdbc935-fe7f-463a-b89e-cad626bdb712"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:36.327897 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.327873    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22a752a7-55f6-4852-8bdc-34ea6a7262a8" (UID: "22a752a7-55f6-4852-8bdc-34ea6a7262a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:36.329335 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.329311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp" (OuterVolumeSpecName: "kube-api-access-j4hqp") pod "f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" (UID: "f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2"). InnerVolumeSpecName "kube-api-access-j4hqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:36.329431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.329339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh" (OuterVolumeSpecName: "kube-api-access-hztsh") pod "22a752a7-55f6-4852-8bdc-34ea6a7262a8" (UID: "22a752a7-55f6-4852-8bdc-34ea6a7262a8"). InnerVolumeSpecName "kube-api-access-hztsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:36.329431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.329359    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22a752a7-55f6-4852-8bdc-34ea6a7262a8" (UID: "22a752a7-55f6-4852-8bdc-34ea6a7262a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:55:36.329431 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.329376    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg" (OuterVolumeSpecName: "kube-api-access-b84hg") pod "9cdbc935-fe7f-463a-b89e-cad626bdb712" (UID: "9cdbc935-fe7f-463a-b89e-cad626bdb712"). InnerVolumeSpecName "kube-api-access-b84hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428699    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b84hg\" (UniqueName: \"kubernetes.io/projected/9cdbc935-fe7f-463a-b89e-cad626bdb712-kube-api-access-b84hg\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428738    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22a752a7-55f6-4852-8bdc-34ea6a7262a8-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428751    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hztsh\" (UniqueName: \"kubernetes.io/projected/22a752a7-55f6-4852-8bdc-34ea6a7262a8-kube-api-access-hztsh\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428762    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428775    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cdbc935-fe7f-463a-b89e-cad626bdb712-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428787    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a752a7-55f6-4852-8bdc-34ea6a7262a8-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.428811 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.428798    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4hqp\" (UniqueName: \"kubernetes.io/projected/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2-kube-api-access-j4hqp\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:36.587595 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.587551    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:55:36.597617 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.597582    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5523b7-7d84848c76-6x8xk"]
Jul 28 18:55:36.620101 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.620068    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:55:36.641676 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.641640    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5523b7-6779645d75-kdwhl"]
Jul 28 18:55:36.661065 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.661034    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:55:36.675764 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.675735    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5523b7-59f7f689bb-sgn2b"]
Jul 28 18:55:36.964394 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.964359    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" path="/var/lib/kubelet/pods/22a752a7-55f6-4852-8bdc-34ea6a7262a8/volumes"
Jul 28 18:55:36.964771 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.964744    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cdbc935-fe7f-463a-b89e-cad626bdb712" path="/var/lib/kubelet/pods/9cdbc935-fe7f-463a-b89e-cad626bdb712/volumes"
Jul 28 18:55:36.965058 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:36.965045    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" path="/var/lib/kubelet/pods/f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2/volumes"
Jul 28 18:55:53.575872 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.575825    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:55:53.576551 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.576122    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" podUID="531b317f-fb7d-4069-ab65-9edf89b7f526" containerName="maas-api" containerID="cri-o://4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec" gracePeriod=30
Jul 28 18:55:53.635317 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.635265    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:55:53.635583 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.635548    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" podUID="b959e62d-cc44-4d16-814c-2db90c91ba54" containerName="payload-processing" containerID="cri-o://8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a" gracePeriod=30
Jul 28 18:55:53.640232 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.639972    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:55:53.640232 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.640196    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" podUID="de89c7b7-b626-4801-83b1-90fb941bdbd8" containerName="payload-pre-processing" containerID="cri-o://33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6" gracePeriod=30
Jul 28 18:55:53.906912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.906878    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:55:53.924347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.924326    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:55:53.932640 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:53.932617    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:55:54.007827 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.007795    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pdg2\" (UniqueName: \"kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2\") pod \"531b317f-fb7d-4069-ab65-9edf89b7f526\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") "
Jul 28 18:55:54.008032 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.007894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np2wv\" (UniqueName: \"kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv\") pod \"b959e62d-cc44-4d16-814c-2db90c91ba54\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") "
Jul 28 18:55:54.008032 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008007    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume\") pod \"de89c7b7-b626-4801-83b1-90fb941bdbd8\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") "
Jul 28 18:55:54.008169 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008064    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume\") pod \"b959e62d-cc44-4d16-814c-2db90c91ba54\" (UID: \"b959e62d-cc44-4d16-814c-2db90c91ba54\") "
Jul 28 18:55:54.008169 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca\") pod \"531b317f-fb7d-4069-ab65-9edf89b7f526\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") "
Jul 28 18:55:54.008169 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx7rw\" (UniqueName: \"kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw\") pod \"de89c7b7-b626-4801-83b1-90fb941bdbd8\" (UID: \"de89c7b7-b626-4801-83b1-90fb941bdbd8\") "
Jul 28 18:55:54.008347 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008186    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") pod \"531b317f-fb7d-4069-ab65-9edf89b7f526\" (UID: \"531b317f-fb7d-4069-ab65-9edf89b7f526\") "
Jul 28 18:55:54.008878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008565    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "531b317f-fb7d-4069-ab65-9edf89b7f526" (UID: "531b317f-fb7d-4069-ab65-9edf89b7f526"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:54.008878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008597    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de89c7b7-b626-4801-83b1-90fb941bdbd8" (UID: "de89c7b7-b626-4801-83b1-90fb941bdbd8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:54.008878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008657    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b959e62d-cc44-4d16-814c-2db90c91ba54" (UID: "b959e62d-cc44-4d16-814c-2db90c91ba54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 18:55:54.008878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008686    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de89c7b7-b626-4801-83b1-90fb941bdbd8-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.008878 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.008705    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/531b317f-fb7d-4069-ab65-9edf89b7f526-openshift-service-ca\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.010208 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.010188    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv" (OuterVolumeSpecName: "kube-api-access-np2wv") pod "b959e62d-cc44-4d16-814c-2db90c91ba54" (UID: "b959e62d-cc44-4d16-814c-2db90c91ba54"). InnerVolumeSpecName "kube-api-access-np2wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:54.010284 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.010230    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2" (OuterVolumeSpecName: "kube-api-access-8pdg2") pod "531b317f-fb7d-4069-ab65-9edf89b7f526" (UID: "531b317f-fb7d-4069-ab65-9edf89b7f526"). InnerVolumeSpecName "kube-api-access-8pdg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:54.010910 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.010878    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "531b317f-fb7d-4069-ab65-9edf89b7f526" (UID: "531b317f-fb7d-4069-ab65-9edf89b7f526"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 18:55:54.010910 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.010881    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw" (OuterVolumeSpecName: "kube-api-access-wx7rw") pod "de89c7b7-b626-4801-83b1-90fb941bdbd8" (UID: "de89c7b7-b626-4801-83b1-90fb941bdbd8"). InnerVolumeSpecName "kube-api-access-wx7rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 18:55:54.109762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.109672    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np2wv\" (UniqueName: \"kubernetes.io/projected/b959e62d-cc44-4d16-814c-2db90c91ba54-kube-api-access-np2wv\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.109762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.109704    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b959e62d-cc44-4d16-814c-2db90c91ba54-plugins-config-volume\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.109762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.109713    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx7rw\" (UniqueName: \"kubernetes.io/projected/de89c7b7-b626-4801-83b1-90fb941bdbd8-kube-api-access-wx7rw\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.109762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.109727    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/531b317f-fb7d-4069-ab65-9edf89b7f526-maas-api-tls\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.109762 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.109736    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pdg2\" (UniqueName: \"kubernetes.io/projected/531b317f-fb7d-4069-ab65-9edf89b7f526-kube-api-access-8pdg2\") on node \"ip-10-0-133-244.ec2.internal\" DevicePath \"\""
Jul 28 18:55:54.341191 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.341155    2581 generic.go:356] "Generic (PLEG): container finished" podID="531b317f-fb7d-4069-ab65-9edf89b7f526" containerID="4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec" exitCode=0
Jul 28 18:55:54.341411 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.341200    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"
Jul 28 18:55:54.341411 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.341242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" event={"ID":"531b317f-fb7d-4069-ab65-9edf89b7f526","Type":"ContainerDied","Data":"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"}
Jul 28 18:55:54.341411 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.341313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf" event={"ID":"531b317f-fb7d-4069-ab65-9edf89b7f526","Type":"ContainerDied","Data":"12a9c68491d65a70afe8a7dbf672ae332f9c4f0eb2ee601fb3505417e31b8d9e"}
Jul 28 18:55:54.341411 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.341340    2581 scope.go:117] "RemoveContainer" containerID="4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"
Jul 28 18:55:54.342713 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.342684    2581 generic.go:356] "Generic (PLEG): container finished" podID="b959e62d-cc44-4d16-814c-2db90c91ba54" containerID="8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a" exitCode=0
Jul 28 18:55:54.342844 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.342791    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"
Jul 28 18:55:54.342844 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.342806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" event={"ID":"b959e62d-cc44-4d16-814c-2db90c91ba54","Type":"ContainerDied","Data":"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"}
Jul 28 18:55:54.342954 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.342851    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt" event={"ID":"b959e62d-cc44-4d16-814c-2db90c91ba54","Type":"ContainerDied","Data":"a2192ae39c247badd32258ae825333c55e9c94e851625f77dcf93301a20cd5b0"}
Jul 28 18:55:54.344300 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.344197    2581 generic.go:356] "Generic (PLEG): container finished" podID="de89c7b7-b626-4801-83b1-90fb941bdbd8" containerID="33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6" exitCode=0
Jul 28 18:55:54.344300 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.344236    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"
Jul 28 18:55:54.344300 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.344250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" event={"ID":"de89c7b7-b626-4801-83b1-90fb941bdbd8","Type":"ContainerDied","Data":"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"}
Jul 28 18:55:54.344507 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.344317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl" event={"ID":"de89c7b7-b626-4801-83b1-90fb941bdbd8","Type":"ContainerDied","Data":"a0ba8df96ccbb56f38e34a99fbd7ad5e328cfbded1c96b7bedc1f12c8bd57642"}
Jul 28 18:55:54.351997 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.351915    2581 scope.go:117] "RemoveContainer" containerID="4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"
Jul 28 18:55:54.352263 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:54.352243    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec\": container with ID starting with 4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec not found: ID does not exist" containerID="4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"
Jul 28 18:55:54.352369 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.352296    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec"} err="failed to get container status \"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec\": rpc error: code = NotFound desc = could not find container \"4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec\": container with ID starting with 4fffd8091b47c63d3ab394e6b49138501e8be1a935326ad9e325eddafb205aec not found: ID does not exist"
Jul 28 18:55:54.352369 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.352320    2581 scope.go:117] "RemoveContainer" containerID="8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"
Jul 28 18:55:54.360239 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.360223    2581 scope.go:117] "RemoveContainer" containerID="8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"
Jul 28 18:55:54.360491 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:54.360473    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a\": container with ID starting with 8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a not found: ID does not exist" containerID="8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"
Jul 28 18:55:54.360556 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.360499    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a"} err="failed to get container status \"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a\": rpc error: code = NotFound desc = could not find container \"8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a\": container with ID starting with 8f63baeeb49daae0dded800122e636f1d09984207e2dccf4ed70b70e22451b1a not found: ID does not exist"
Jul 28 18:55:54.360556 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.360517    2581 scope.go:117] "RemoveContainer" containerID="33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"
Jul 28 18:55:54.368599 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.368577    2581 scope.go:117] "RemoveContainer" containerID="33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"
Jul 28 18:55:54.368913 ip-10-0-133-244 kubenswrapper[2581]: E0728 18:55:54.368895    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6\": container with ID starting with 33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6 not found: ID does not exist" containerID="33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"
Jul 28 18:55:54.368978 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.368919    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6"} err="failed to get container status \"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6\": rpc error: code = NotFound desc = could not find container \"33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6\": container with ID starting with 33f7ffdab10d0af568cefdc1d8f734ad391cbaf8df4612d9acb4a558e3fbc9b6 not found: ID does not exist"
Jul 28 18:55:54.370028 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.370002    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:55:54.375015 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.374995    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-10fb8f-54d4875457-dwmqf"]
Jul 28 18:55:54.386923 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.386898    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:55:54.394499 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.394477    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-10fb8f-7d5649dd4b-5g5bt"]
Jul 28 18:55:54.405534 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.405510    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:55:54.412653 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.412631    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-10fb8f-5f8959b6fb-446rl"]
Jul 28 18:55:54.964951 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.964916    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="531b317f-fb7d-4069-ab65-9edf89b7f526" path="/var/lib/kubelet/pods/531b317f-fb7d-4069-ab65-9edf89b7f526/volumes"
Jul 28 18:55:54.965366 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.965302    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b959e62d-cc44-4d16-814c-2db90c91ba54" path="/var/lib/kubelet/pods/b959e62d-cc44-4d16-814c-2db90c91ba54/volumes"
Jul 28 18:55:54.965620 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:55:54.965607    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de89c7b7-b626-4801-83b1-90fb941bdbd8" path="/var/lib/kubelet/pods/de89c7b7-b626-4801-83b1-90fb941bdbd8/volumes"
Jul 28 18:56:06.852881 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:06.852843    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5655468897-x4bbh_2dc9ffb5-2f64-4c1f-b551-6b71479c5292/manager/0.log"
Jul 28 18:56:07.458257 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:07.458231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-nw82x_5763e90e-e15e-4151-a35d-54d3702b0143/copy-manifests/0.log"
Jul 28 18:56:07.471148 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:07.471120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-nw82x_5763e90e-e15e-4151-a35d-54d3702b0143/manager/0.log"
Jul 28 18:56:07.842045 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:07.842014    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d85b9d6c-knnsr_c77f22cb-2a92-4457-921f-11444f9b5235/maas-api/0.log"
Jul 28 18:56:09.797901 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:09.797871    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qps2q_1ebd22e3-a69d-4369-9c30-148d0f8142f4/manager/0.log"
Jul 28 18:56:11.014491 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:11.014458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nd6xq_1078f0fb-b4de-46b8-a2eb-ef18c5255fab/discovery/0.log"
Jul 28 18:56:11.487743 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:11.487714    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vs95b_4777917d-cf52-4181-a872-d7e46ff6a32a/payload-pre-processing/0.log"
Jul 28 18:56:12.090586 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:12.090554    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tn2br_a3e580e0-0dcf-4079-b634-4d0e6714d026/main/0.log"
Jul 28 18:56:12.428129 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:12.428048    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fb9wl5_d5eb9f97-0841-4598-a2bb-5f2e5788760c/main/0.log"
Jul 28 18:56:20.907695 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:20.907655    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8lwbk_bcfd65e2-69dc-4593-aa86-aa6a8c84c366/global-pull-secret-syncer/0.log"
Jul 28 18:56:21.140686 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:21.140649    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m9m2v_15fec211-1790-4736-a469-069077d4c7fa/konnectivity-agent/0.log"
Jul 28 18:56:21.195922 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:21.195839    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-244.ec2.internal_505a36e67b1ce4359d0b61ae9f5542fa/haproxy/0.log"
Jul 28 18:56:27.201387 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:27.201261    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qps2q_1ebd22e3-a69d-4369-9c30-148d0f8142f4/manager/0.log"
Jul 28 18:56:29.683515 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.683468    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-t7586_ff634b17-0e6a-456b-b337-2db00422219d/cluster-monitoring-operator/0.log"
Jul 28 18:56:29.717539 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.717509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tnftj_a9f678d9-85df-4585-96c1-20086735ff25/kube-state-metrics/0.log"
Jul 28 18:56:29.745072 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.745044    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tnftj_a9f678d9-85df-4585-96c1-20086735ff25/kube-rbac-proxy-main/0.log"
Jul 28 18:56:29.768817 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.768789    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-tnftj_a9f678d9-85df-4585-96c1-20086735ff25/kube-rbac-proxy-self/0.log"
Jul 28 18:56:29.803362 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.803335    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-586fc8b7fd-rp9j8_40a11f41-e839-46a6-911f-4af0ee4650c7/metrics-server/0.log"
Jul 28 18:56:29.830482 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.830449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-mbxq4_215fc5e1-f230-4ebc-a3a6-7690653dc547/monitoring-plugin/0.log"
Jul 28 18:56:29.947844 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.947757    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzfmg_accd0561-b1ee-4799-a0de-6041f81b59b1/node-exporter/0.log"
Jul 28 18:56:29.971797 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.971768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzfmg_accd0561-b1ee-4799-a0de-6041f81b59b1/kube-rbac-proxy/0.log"
Jul 28 18:56:29.995490 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:29.995466    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzfmg_accd0561-b1ee-4799-a0de-6041f81b59b1/init-textfile/0.log"
Jul 28 18:56:30.212318 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.212237    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/prometheus/0.log"
Jul 28 18:56:30.236181 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.236158    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/config-reloader/0.log"
Jul 28 18:56:30.261786 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.261762    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/thanos-sidecar/0.log"
Jul 28 18:56:30.288953 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.288920    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/kube-rbac-proxy-web/0.log"
Jul 28 18:56:30.316868 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.316842    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/kube-rbac-proxy/0.log"
Jul 28 18:56:30.342453 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.342425    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/kube-rbac-proxy-thanos/0.log"
Jul 28 18:56:30.366221 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:30.366189    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4c4f9f9d-c1eb-4e0e-8cb5-ff8e53c33003/init-config-reloader/0.log"
Jul 28 18:56:31.414845 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.414811    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"]
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415154    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee71262-8970-4f78-85c9-0a389ec20fd6" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415165    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee71262-8970-4f78-85c9-0a389ec20fd6" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415180    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" containerName="maas-api"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415186    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" containerName="maas-api"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415192    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de89c7b7-b626-4801-83b1-90fb941bdbd8" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415197    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="de89c7b7-b626-4801-83b1-90fb941bdbd8" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415210    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="531b317f-fb7d-4069-ab65-9edf89b7f526" containerName="maas-api"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415215    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="531b317f-fb7d-4069-ab65-9edf89b7f526" containerName="maas-api"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415222    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c740f049-8c04-44e9-942d-c237649ea780" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415228    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c740f049-8c04-44e9-942d-c237649ea780" containerName="payload-pre-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415236    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="683f477c-efdb-44f7-9723-5c779ceba872" containerName="payload-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415243    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="683f477c-efdb-44f7-9723-5c779ceba872" containerName="payload-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415249    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" containerName="payload-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415254    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" containerName="payload-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415260    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b959e62d-cc44-4d16-814c-2db90c91ba54" containerName="payload-processing"
Jul 28 18:56:31.415258 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415265    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b959e62d-cc44-4d16-814c-2db90c91ba54" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415286    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" containerName="maas-api"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415291    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" containerName="maas-api"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415296    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415301    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415313    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f70b1ec-9d3e-4996-905b-68f212655c2b" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415318    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f70b1ec-9d3e-4996-905b-68f212655c2b" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415325    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cdbc935-fe7f-463a-b89e-cad626bdb712" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415330    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cdbc935-fe7f-463a-b89e-cad626bdb712" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415383    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a752a7-55f6-4852-8bdc-34ea6a7262a8" containerName="maas-api"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415390    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f70b1ec-9d3e-4996-905b-68f212655c2b" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="de89c7b7-b626-4801-83b1-90fb941bdbd8" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415402    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="531b317f-fb7d-4069-ab65-9edf89b7f526" containerName="maas-api"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415408    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b959e62d-cc44-4d16-814c-2db90c91ba54" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415414    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="683f477c-efdb-44f7-9723-5c779ceba872" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415419    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee71262-8970-4f78-85c9-0a389ec20fd6" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415425    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cdbc935-fe7f-463a-b89e-cad626bdb712" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415430    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f19f4ebe-e5d9-455d-a6ff-05b11b5a74a2" containerName="payload-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415434    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="570f8320-82f0-4841-b6f8-4e3ef5d9cdce" containerName="maas-api"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415439    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c740f049-8c04-44e9-942d-c237649ea780" containerName="payload-pre-processing"
Jul 28 18:56:31.415831 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.415444    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7c1ad2-ea0a-41be-ab44-c787f517eaa0" containerName="payload-processing"
Jul 28 18:56:31.418504 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.418488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.420933 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.420885    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xtqp2\"/\"kube-root-ca.crt\""
Jul 28 18:56:31.421084 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.421030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xtqp2\"/\"default-dockercfg-rdvgc\""
Jul 28 18:56:31.421932 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.421897    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xtqp2\"/\"openshift-service-ca.crt\""
Jul 28 18:56:31.428707 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.428684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"]
Jul 28 18:56:31.558761 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.558727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-sys\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.558945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.558775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2zpr\" (UniqueName: \"kubernetes.io/projected/b5440161-7088-469f-bf37-a4f95e7d753d-kube-api-access-z2zpr\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.558945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.558832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-podres\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.558945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.558870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-lib-modules\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.558945 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.558907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-proc\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2zpr\" (UniqueName: \"kubernetes.io/projected/b5440161-7088-469f-bf37-a4f95e7d753d-kube-api-access-z2zpr\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660172 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-podres\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-lib-modules\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-proc\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-sys\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-podres\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.660432 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-sys\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.665344 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-lib-modules\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.665344 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.660962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b5440161-7088-469f-bf37-a4f95e7d753d-proc\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.669463 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.669437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2zpr\" (UniqueName: \"kubernetes.io/projected/b5440161-7088-469f-bf37-a4f95e7d753d-kube-api-access-z2zpr\") pod \"perf-node-gather-daemonset-jqjqj\" (UID: \"b5440161-7088-469f-bf37-a4f95e7d753d\") " pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.729098 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.729063    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:31.867202 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:31.867174    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"]
Jul 28 18:56:31.869416 ip-10-0-133-244 kubenswrapper[2581]: W0728 18:56:31.869390    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5440161_7088_469f_bf37_a4f95e7d753d.slice/crio-fb351c4c83589cc62eae8a152f3764d4b4281fad4c1c889c377260bd26614df7 WatchSource:0}: Error finding container fb351c4c83589cc62eae8a152f3764d4b4281fad4c1c889c377260bd26614df7: Status 404 returned error can't find the container with id fb351c4c83589cc62eae8a152f3764d4b4281fad4c1c889c377260bd26614df7
Jul 28 18:56:32.526795 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:32.526757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj" event={"ID":"b5440161-7088-469f-bf37-a4f95e7d753d","Type":"ContainerStarted","Data":"62bf5367d9f4c931b179c2711a6467cf2087e71d2d553c718d276c581524c3ed"}
Jul 28 18:56:32.527398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:32.526852    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:32.527398 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:32.526873    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj" event={"ID":"b5440161-7088-469f-bf37-a4f95e7d753d","Type":"ContainerStarted","Data":"fb351c4c83589cc62eae8a152f3764d4b4281fad4c1c889c377260bd26614df7"}
Jul 28 18:56:32.543531 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:32.543481    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj" podStartSLOduration=1.543466116 podStartE2EDuration="1.543466116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:56:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:56:32.54240881 +0000 UTC m=+3572.112471643" watchObservedRunningTime="2026-07-28 18:56:32.543466116 +0000 UTC m=+3572.113528946"
Jul 28 18:56:34.731433 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:34.731408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7k8k8_35236c73-366b-4c06-8832-56a0790f2282/dns/0.log"
Jul 28 18:56:34.756247 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:34.756219    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7k8k8_35236c73-366b-4c06-8832-56a0790f2282/kube-rbac-proxy/0.log"
Jul 28 18:56:34.882133 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:34.882100    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kc972_7ac26091-4d06-4f7a-bcf7-acde7784cc74/dns-node-resolver/0.log"
Jul 28 18:56:35.505122 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:35.505088    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8f69b6c-vm7sw_87658cb9-e17e-40cf-8beb-10cf0a888659/registry/0.log"
Jul 28 18:56:35.587912 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:35.587884    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sl47n_2e67f7db-91a9-4e7f-bfcc-21b79228466d/node-ca/0.log"
Jul 28 18:56:36.759311 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:36.759258    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nd6xq_1078f0fb-b4de-46b8-a2eb-ef18c5255fab/discovery/0.log"
Jul 28 18:56:36.934616 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:36.934585    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vs95b_4777917d-cf52-4181-a872-d7e46ff6a32a/payload-pre-processing/0.log"
Jul 28 18:56:37.821284 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:37.821245    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g2hl8_8dea7d38-b4c7-49a8-80a3-c86f9e10e06b/serve-healthcheck-canary/0.log"
Jul 28 18:56:38.392229 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.392170    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zhnzc_631859d9-19f5-4e56-9af5-cf1f161e40fd/insights-operator/0.log"
Jul 28 18:56:38.394639 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.394620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zhnzc_631859d9-19f5-4e56-9af5-cf1f161e40fd/insights-operator/1.log"
Jul 28 18:56:38.419407 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.419385    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ndmx_f98f8ecb-0659-442b-9b35-eefff9f4007d/kube-rbac-proxy/0.log"
Jul 28 18:56:38.444480 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.444445    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ndmx_f98f8ecb-0659-442b-9b35-eefff9f4007d/exporter/0.log"
Jul 28 18:56:38.468245 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.468218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ndmx_f98f8ecb-0659-442b-9b35-eefff9f4007d/extractor/0.log"
Jul 28 18:56:38.542168 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:38.542143    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xtqp2/perf-node-gather-daemonset-jqjqj"
Jul 28 18:56:41.642503 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:41.642462    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5655468897-x4bbh_2dc9ffb5-2f64-4c1f-b551-6b71479c5292/manager/0.log"
Jul 28 18:56:41.992913 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:41.992821    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-nw82x_5763e90e-e15e-4151-a35d-54d3702b0143/manager/0.log"
Jul 28 18:56:42.010784 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:42.010756    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fb95c4cb9-nw82x_5763e90e-e15e-4151-a35d-54d3702b0143/copy-manifests/0.log"
Jul 28 18:56:42.506257 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:42.506222    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nd6xq_1078f0fb-b4de-46b8-a2eb-ef18c5255fab/discovery/0.log"
Jul 28 18:56:42.650043 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:42.649961    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vs95b_4777917d-cf52-4181-a872-d7e46ff6a32a/payload-pre-processing/0.log"
Jul 28 18:56:43.345491 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:43.345461    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cd6475d44-q42r7_1111f638-c9a8-4f15-affa-6cddc24d3f43/manager/0.log"
Jul 28 18:56:44.408249 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:44.408208    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-nd6xq_1078f0fb-b4de-46b8-a2eb-ef18c5255fab/discovery/0.log"
Jul 28 18:56:44.539796 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:44.539765    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vs95b_4777917d-cf52-4181-a872-d7e46ff6a32a/payload-pre-processing/0.log"
Jul 28 18:56:48.314114 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:48.314085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x9mr4_4388985f-f081-4bae-b679-3b9b1f1fefd1/migrator/0.log"
Jul 28 18:56:48.335991 ip-10-0-133-244 kubenswrapper[2581]: I0728 18:56:48.335961    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-x9mr4_4388985f-f081-4bae-b679-3b9b1f1fefd1/graceful-termination/0.log"