Jul 24 19:34:28.792511 ip-10-0-129-222 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 19:34:29.170252 ip-10-0-129-222 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 19:34:29.170252 ip-10-0-129-222 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 19:34:29.170252 ip-10-0-129-222 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 19:34:29.170252 ip-10-0-129-222 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 19:34:29.170252 ip-10-0-129-222 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 19:34:29.171897 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.171789    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 19:34:29.180109 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180085    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.180109 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180110    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.180109 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180115    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180120    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180124    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180128    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180132    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180136    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180140    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180144    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180148    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180152    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180156    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180165    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180169    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180172    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180178    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180185    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180189    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180193    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180206    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180211    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.180336 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180215    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180219    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180223    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180228    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180237    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180244    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180250    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180272    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180279    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180283    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180288    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180293    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180298    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180303    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180310    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180315    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180324    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180329    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180333    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.181131 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180338    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180342    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180346    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180351    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180355    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180360    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180364    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180369    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180373    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180384    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180389    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180394    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180407    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180412    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180417    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180422    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180426    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180431    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180435    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180440    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.181631 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180444    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180449    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180458    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180462    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180467    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180471    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180475    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180481    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180486    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180490    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180495    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180499    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180504    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180518    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180522    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180532    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180537    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180541    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180546    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180550    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.182122 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180554    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180559    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180573    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180578    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180583    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180598    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180607    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180614    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180621    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180626    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180631    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180636    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180640    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180646    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180651    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180656    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180660    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.180665    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181731    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.182635 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181743    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181747    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181750    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181753    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181756    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181758    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181761    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181763    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181767    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181769    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181772    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181774    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181777    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181780    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181782    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181785    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181787    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181790    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181793    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181796    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.183147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181799    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181801    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181804    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181807    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181809    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181812    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181814    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181817    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181819    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181822    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181824    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181827    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181831    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181833    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181836    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181840    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181844    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181847    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181850    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181852    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.183676 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181855    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181857    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181860    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181862    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181865    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181868    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181870    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181873    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181876    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181878    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181881    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181884    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181887    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181890    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181893    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181895    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181898    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181901    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181903    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181906    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.184163 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181908    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181911    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181914    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181916    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181919    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181922    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181925    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181928    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181932    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181936    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181941    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181944    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181948    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181951    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181954    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181957    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181959    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181962    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181965    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.184659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181968    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181970    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181973    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181976    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181979    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181981    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181984    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181986    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181989    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181991    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181994    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181996    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.181999    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182004    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182007    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182010    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182012    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182015    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182018    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182423    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.185176 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182428    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182433    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182436    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182439    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182442    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182445    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182448    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182450    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182453    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182456    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182458    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182461    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182464    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182466    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182469    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182472    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182475    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182477    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182480    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.185678 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182482    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182485    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182488    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182490    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182493    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182496    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182500    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182503    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182505    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182508    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182511    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182513    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182516    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182518    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182521    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182524    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182526    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182529    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182532    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182534    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182537    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.186147 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182540    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182542    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182545    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182547    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182550    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182552    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182555    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182557    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182561    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182564    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182566    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182569    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182572    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182574    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182576    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182579    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182582    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182586    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182588    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.186683 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182591    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182593    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182596    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182599    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182601    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182604    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182607    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182609    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182612    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182614    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182617    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182620    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182622    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182625    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182627    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182630    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182632    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182635    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182637    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182640    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.187139 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182644    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182647    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182650    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182654    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182657    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182659    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182662    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182664    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182667    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182669    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182673    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182676    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182678    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182681    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182686    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182690    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182693    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182696    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.182699    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182772    2568 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 19:34:29.187702 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182779    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182785    2568 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182789    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182793    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182796    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182800    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182804    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182808    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182811    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182814    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182817    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182820    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182823    2568 flags.go:64] FLAG: --cgroup-root=""
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182826    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182829    2568 flags.go:64] FLAG: --client-ca-file=""
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182833    2568 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182836    2568 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182841    2568 flags.go:64] FLAG: --cluster-domain=""
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182843    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182847    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182850    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182853    2568 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182858    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 19:34:29.188193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182861    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182864    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182867    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182871    2568 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182874    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182877    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182880    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182883    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182887    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182890    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182893    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182896    2568 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182899    2568 flags.go:64] FLAG: --enable-server="true"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182902    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182906    2568 flags.go:64] FLAG: --event-burst="100"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182909    2568 flags.go:64] FLAG: --event-qps="50"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182912    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182915    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182917    2568 flags.go:64] FLAG: --eviction-hard=""
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182921    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182924    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182926    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182930    2568 flags.go:64] FLAG: --eviction-soft=""
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182933    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182936    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 19:34:29.188763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182939    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182942    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182945    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182948    2568 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182951    2568 flags.go:64] FLAG: --feature-gates=""
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182954    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182958    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182961    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182964    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182967    2568 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182972    2568 flags.go:64] FLAG: --help="false"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182975    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182978    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182981    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182984    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182987    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182990    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182993    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182996    2568 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.182999    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183002    2568 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183005    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183008    2568 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183011    2568 flags.go:64] FLAG: --kube-reserved=""
Jul 24 19:34:29.189424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183014    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183021    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183024    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183027    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183030    2568 flags.go:64] FLAG: --lock-file=""
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183033    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183036    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183039    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183045    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183049    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183052    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183055    2568 flags.go:64] FLAG: --logging-format="text"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183058    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183061    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183065    2568 flags.go:64] FLAG: --manifest-url=""
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183068    2568 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183072    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183075    2568 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183080    2568 flags.go:64] FLAG: --max-pods="110"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183083    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183086    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183089    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183092    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183095    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183098    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 19:34:29.190020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183101    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183108    2568 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183111    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183114    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183117    2568 flags.go:64] FLAG: --pod-cidr=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183120    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183125    2568 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183128    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183132    2568 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183134    2568 flags.go:64] FLAG: --port="10250"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183138    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183141    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b57e552fe57860b1"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183144    2568 flags.go:64] FLAG: --qos-reserved=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183147    2568 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183150    2568 flags.go:64] FLAG: --register-node="true"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183153    2568 flags.go:64] FLAG: --register-with-taints=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183156    2568 flags.go:64] FLAG: --registry-burst="10"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183160    2568 flags.go:64] FLAG: --registry-qps="5"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183163    2568 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183166    2568 flags.go:64] FLAG: --reserved-memory=""
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183169    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183174    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183177    2568 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183179    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183182    2568 flags.go:64] FLAG: --runonce="false"
Jul 24 19:34:29.190692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183185    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183189    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183192    2568 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183195    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183198    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183201    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183204    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183207    2568 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183210    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183213    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183216    2568 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183219    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183221    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183224    2568 flags.go:64] FLAG: --system-cgroups=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183227    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183232    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183235    2568 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183238    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183242    2568 flags.go:64] FLAG: --tls-min-version=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183245    2568 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183247    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183250    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183253    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183270    2568 flags.go:64] FLAG: --v="2"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183274    2568 flags.go:64] FLAG: --version="false"
Jul 24 19:34:29.191352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183279    2568 flags.go:64] FLAG: --vmodule=""
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183283    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.183286    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183406    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183410    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183413    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183416    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183419    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183423    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183426    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183429    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183431    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183434    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183437    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183441    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183445    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183447    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183450    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183453    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183456    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.191952 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183459    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183462    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183465    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183468    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183470    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183473    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183476    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183478    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183481    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183483    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183486    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183489    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183491    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183495    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183497    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183501    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183503    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183506    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183509    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183511    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.192467 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183516    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183518    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183521    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183524    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183526    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183530    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183532    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183535    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183537    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183540    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183542    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183545    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183547    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183550    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183553    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183555    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183558    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183560    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183563    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.192953 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183565    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183569    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183573    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183576    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183579    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183582    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183584    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183588    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183592    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183596    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183600    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183603    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183606    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183610    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183613    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183615    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183619    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183621    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183624    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.193440 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183626    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183629    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183632    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183634    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183637    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183640    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183642    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183645    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183648    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183650    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183653    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183655    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183658    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183660    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183663    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183665    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183668    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183671    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183673    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183676    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.193920 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183678    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183683    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183686    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.183689    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.184253    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.191188    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.191205    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.191224    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.191233    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191313    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191318    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191321    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191324    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191327    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.194444 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191331    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191335    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191339    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191343    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191346    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191348    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191351    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191354    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191358    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191361    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191363    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191368    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191372    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191377    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191380    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191383    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191386    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191390    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191393    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.194795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191396    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191399    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191401    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191404    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191406    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191410    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191413    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191416    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191419    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191421    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191424    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191427    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191430    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191433    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191435    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191438    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191440    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191443    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191446    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.195274 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191448    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191451    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191453    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191456    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191459    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191462    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191465    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191468    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191470    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191473    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191476    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191479    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191481    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191483    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191486    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191489    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191491    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191494    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191498    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191501    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.195741 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191503    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191506    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191509    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191511    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191514    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191517    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191519    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191522    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191525    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191528    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191530    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191533    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191535    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191538    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191540    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191543    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191546    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191549    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191551    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191554    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.196225 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191556    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191559    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191562    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191564    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191567    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191570    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191572    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191575    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191577    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191580    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191583    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191585    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191588    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191591    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191593    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.196795 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191596    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.191601    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191732    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191736    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191739    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191742    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191744    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191747    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191750    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191753    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191756    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191758    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191761    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191764    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191767    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 19:34:29.197197 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191770    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191774    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191777    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191780    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191783    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191785    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191788    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191790    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191793    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191796    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191798    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191801    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191803    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191806    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191808    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191811    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191814    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191816    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191819    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191821    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 19:34:29.197606 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191824    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191827    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191829    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191832    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191835    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191837    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191840    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191843    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191845    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191848    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191850    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191853    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191856    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191858    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191861    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191864    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191866    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191869    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191871    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 19:34:29.198094 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191874    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191876    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191879    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191881    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191884    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191886    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191889    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191892    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191894    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191897    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191900    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191902    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191905    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191907    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191910    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191912    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191915    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191917    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191920    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191922    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 19:34:29.198568 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191925    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191927    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191930    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191934    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191937    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191940    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191943    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191946    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191948    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191951    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191954    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191957    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191960    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191963    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191965    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191968    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191973    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191976    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191979    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 19:34:29.199057 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191982    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191984    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191987    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191991    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191994    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191996    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.191999    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:29.192002    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.192007    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.192122    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.194047    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.195342    2568 server.go:1019] "Starting client certificate rotation"
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.195453    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 19:34:29.199630 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.195482    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 19:34:29.199956 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.197555    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 19:34:29.199956 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.199277    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 19:34:29.211944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.211923    2568 log.go:25] "Validated CRI v1 runtime API"
Jul 24 19:34:29.213891 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.213878    2568 log.go:25] "Validated CRI v1 image API"
Jul 24 19:34:29.214740 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.214727    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 19:34:29.219034 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.219012    2568 fs.go:135] Filesystem UUIDs: map[3c340002-aff7-40f6-9497-079b9123b479:/dev/nvme0n1p4 469d0baf-8045-4ada-ae0d-868ce30476fa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 24 19:34:29.219097 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.219032    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 19:34:29.221151 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.221134    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 19:34:29.223971 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.223862    2568 manager.go:216] Machine: {Timestamp:2026-07-24 19:34:29.22260352 +0000 UTC m=+0.317149124 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099740 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29af733d1bb4323b356f468798303c SystemUUID:ec29af73-3d1b-b432-3b35-6f468798303c BootID:f08a1983-779d-42d1-9bdb-61937d2c7866 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:59:6d:c4:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:59:6d:c4:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:ee:b8:7c:61:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 19:34:29.223971 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.223965    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 19:34:29.224069 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.224036    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 19:34:29.226906 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.226885    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 19:34:29.227052 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.226909    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-222.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 19:34:29.228119 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.228109    2568 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 19:34:29.228155 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.228122    2568 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 19:34:29.228155 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.228129    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 19:34:29.228850 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.228837    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 19:34:29.229692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.229682    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 19:34:29.231321 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231310    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 19:34:29.231422 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231413    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 19:34:29.231453 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231442    2568 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 19:34:29.231453 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231452    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 19:34:29.231510 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231469    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 19:34:29.231510 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231483    2568 kubelet.go:398] "Adding apiserver pod source"
Jul 24 19:34:29.231510 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231492    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 19:34:29.231834 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231823    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 19:34:29.231871 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.231843    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 19:34:29.235523 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.235508    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 19:34:29.237306 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.237291    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 19:34:29.237355 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.237323    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 19:34:29.238733 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238717    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 19:34:29.238733 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238736    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238743    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238751    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238765    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238771    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238782    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238788    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238794    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238801    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238821    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 19:34:29.238864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.238827    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 19:34:29.239611 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.239601    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 19:34:29.239611 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.239611    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 19:34:29.242543 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.242524    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxw4f"
Jul 24 19:34:29.242641 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.242629    2568 server.go:1268] "Started kubelet"
Jul 24 19:34:29.242809 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.242714    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 19:34:29.243317 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.243131    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 19:34:29.243484 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.243468    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 19:34:29.243538 ip-10-0-129-222 systemd[1]: Started Kubernetes Kubelet.
Jul 24 19:34:29.245011 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.244985    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 19:34:29.245476 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.245419    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 19:34:29.245630 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.245578    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 19:34:29.246054 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.246003    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-222.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 19:34:29.246269 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.246241    2568 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 19:34:29.248128 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.248105    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 19:34:29.248375 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.248360    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxw4f"
Jul 24 19:34:29.250361 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.249295    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-222.ec2.internal.18c550cd1f1a45a5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-222.ec2.internal,UID:ip-10-0-129-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-222.ec2.internal,},FirstTimestamp:2026-07-24 19:34:29.242602917 +0000 UTC m=+0.337148522,LastTimestamp:2026-07-24 19:34:29.242602917 +0000 UTC m=+0.337148522,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-222.ec2.internal,}"
Jul 24 19:34:29.253602 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.253582    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 19:34:29.253683 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.253605    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 19:34:29.253754 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.253686    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.254467 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254452    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 19:34:29.254557 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254524    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 19:34:29.254557 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254540    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 19:34:29.254686 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254672    2568 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 19:34:29.254794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254778    2568 factory.go:55] Registering systemd factory
Jul 24 19:34:29.254864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254797    2568 factory.go:223] Registration of the systemd container factory successfully
Jul 24 19:34:29.254864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.254783    2568 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 19:34:29.255043 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.255028    2568 factory.go:153] Registering CRI-O factory
Jul 24 19:34:29.255081 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.255049    2568 factory.go:223] Registration of the crio container factory successfully
Jul 24 19:34:29.255109 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.255102    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 19:34:29.255138 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.255129    2568 factory.go:103] Registering Raw factory
Jul 24 19:34:29.255165 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.255144    2568 manager.go:1196] Started watching for new ooms in manager
Jul 24 19:34:29.256120 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.256102    2568 manager.go:349] Starting recovery of all containers
Jul 24 19:34:29.261326 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.261183    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:29.265386 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.265371    2568 manager.go:354] Recovery completed
Jul 24 19:34:29.265846 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.265827    2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-222.ec2.internal\" not found" node="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.269172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.269160    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.270909 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.270896    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.270986 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.270927    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.270986 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.270943    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.271531 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.271514    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 19:34:29.271531 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.271528    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 19:34:29.271652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.271537    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.271652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.271548    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 19:34:29.272311 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.272294    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.272419 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.272321    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.272419 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.272331    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.273317 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.273304    2568 policy_none.go:49] "None policy: Start"
Jul 24 19:34:29.273363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.273319    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 19:34:29.273363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.273330    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 19:34:29.274294 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.274283    2568 policy_none.go:47] "Start"
Jul 24 19:34:29.309959 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.309796    2568 manager.go:337] "Starting Device Plugin manager"
Jul 24 19:34:29.310064 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.309984    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 19:34:29.310064 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.309998    2568 server.go:87] "Starting device plugin registration server"
Jul 24 19:34:29.310305 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.310293    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 19:34:29.310337 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.310308    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 19:34:29.310438 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.310413    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 19:34:29.310511 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.310492    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 19:34:29.310511 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.310508    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 19:34:29.311024 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.311004    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 19:34:29.311108 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.311058    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.412135 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.412103    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.413363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.413344    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.413459 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.413376    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.413459 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.413389    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.413459 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.413422    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.422476 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.422417    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.422476 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.422444    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-222.ec2.internal\": node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.422668 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.422647    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 19:34:29.423990 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.423971    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 19:34:29.424081 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.424012    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 19:34:29.424081 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.424039    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 19:34:29.424188 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.424081    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 19:34:29.426425 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.426410    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:29.467768 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.467744    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.528455 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.528426    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"]
Jul 24 19:34:29.528534 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.528498    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.530394 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.530377    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.530478 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.530412    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.530478 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.530424    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.531884 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.531872    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.532045 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532030    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.532098 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532072    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.532655 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532626    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.532655 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532647    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.532805 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532676    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.532805 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532681    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.532805 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532692    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.532805 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.532693    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.534014 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.533998    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.534071 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.534027    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 19:34:29.536936 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.536912    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 19:34:29.537045 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.536951    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 19:34:29.537045 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.536980    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeHasSufficientPID"
Jul 24 19:34:29.556777 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.556753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.556860 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.556779    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.556860 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.556799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e46ae8c13e8686631be2da87086a49c5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-222.ec2.internal\" (UID: \"e46ae8c13e8686631be2da87086a49c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.567039 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.567014    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-222.ec2.internal\" not found" node="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.567882 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.567863    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.571651 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.571626    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-222.ec2.internal\" not found" node="ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657701    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657897 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e46ae8c13e8686631be2da87086a49c5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-222.ec2.internal\" (UID: \"e46ae8c13e8686631be2da87086a49c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657897 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657788    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e46ae8c13e8686631be2da87086a49c5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-222.ec2.internal\" (UID: \"e46ae8c13e8686631be2da87086a49c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657897 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657815    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.657897 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.657826    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe59cc5e90685ef84de762ad6b19adee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal\" (UID: \"fe59cc5e90685ef84de762ad6b19adee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.668563 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.668539    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.768824 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.768749    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.869813 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.869771    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:29.869813 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.869816    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.875095 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:29.875072    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:29.969955 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:29.969911    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:30.071054 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:30.070991    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:30.171595 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:30.171564    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-222.ec2.internal\" not found"
Jul 24 19:34:30.195029 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.195002    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 19:34:30.195219 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.195197    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 19:34:30.195311 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.195204    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 19:34:30.252796 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.252756    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 19:29:30 +0000 UTC" deadline="2028-01-25 06:42:23.652324076 +0000 UTC"
Jul 24 19:34:30.252796 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.252786    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13187h7m53.399542104s"
Jul 24 19:34:30.256932 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.256906    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:30.259139 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.259121    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:30.357148 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.357066    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:30.369393 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.369363    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 19:34:30.370823 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.370808    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal"
Jul 24 19:34:30.378480 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.378461    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 19:34:30.573410 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:30.573376    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46ae8c13e8686631be2da87086a49c5.slice/crio-dc8d9e4fcc92b59563726bbaec1bfec72189e7380f049a529a2259e1d4e5590f WatchSource:0}: Error finding container dc8d9e4fcc92b59563726bbaec1bfec72189e7380f049a529a2259e1d4e5590f: Status 404 returned error can't find the container with id dc8d9e4fcc92b59563726bbaec1bfec72189e7380f049a529a2259e1d4e5590f
Jul 24 19:34:30.573638 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:30.573616    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe59cc5e90685ef84de762ad6b19adee.slice/crio-9458e3428e3685aff8b90ed3f02c0f61fb8e298978a748da9d4faecdd14adb70 WatchSource:0}: Error finding container 9458e3428e3685aff8b90ed3f02c0f61fb8e298978a748da9d4faecdd14adb70: Status 404 returned error can't find the container with id 9458e3428e3685aff8b90ed3f02c0f61fb8e298978a748da9d4faecdd14adb70
Jul 24 19:34:30.579004 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:30.578987    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:34:31.046992 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.046962    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:31.234828 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.234785    2568 apiserver.go:52] "Watching apiserver"
Jul 24 19:34:31.245198 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.245167    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 19:34:31.246925 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.246896    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pczcr","kube-system/konnectivity-agent-rbnml","kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj","openshift-cluster-node-tuning-operator/tuned-9f68w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal","openshift-network-diagnostics/network-check-target-cg6jk","openshift-network-operator/iptables-alerter-scmt7","openshift-dns/node-resolver-9dtbm","openshift-image-registry/node-ca-jmnn5","openshift-multus/multus-additional-cni-plugins-nxrpz","openshift-multus/multus-ndh4r","openshift-multus/network-metrics-daemon-8mr59","openshift-ovn-kubernetes/ovnkube-node-ngzdn"]
Jul 24 19:34:31.254469 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.252652    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:31.254469 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.252753    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:31.254649 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.254499    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 19:34:31.255890 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.255771    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.255890 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.255874    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:31.257422 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.257115    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.259295 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.258665    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.260018 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.259996    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.260703 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.260677    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 19:34:31.260808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.260700    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jr95r\""
Jul 24 19:34:31.261354 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.261334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.261974 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.261956    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 19:34:31.262062 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.261992    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 19:34:31.262247 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262224    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.262247 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262208    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.262570 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262444    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 19:34:31.262570 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262462    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.262717 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262675    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 19:34:31.262776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262764    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p2fh2\""
Jul 24 19:34:31.262836 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262745    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.262889 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262855    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.263026 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.262993    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.263096 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.263029    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fpjtv\""
Jul 24 19:34:31.264800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.264058    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.264800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.264483    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.264800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.264612    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.264800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.264674    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 19:34:31.264800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.264704    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gmvcn\""
Jul 24 19:34:31.265724 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.265702    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.265826 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.265765    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:31.267203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.267134    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lpf67\""
Jul 24 19:34:31.267300 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.267139    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.267619 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.267461    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.269617 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.269594    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 19:34:31.270033 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.269922    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z2d7\" (UniqueName: \"kubernetes.io/projected/52a08db7-be3f-4307-a4eb-be1869fcd7b9-kube-api-access-5z2d7\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.270033 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.269961    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:31.270033 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.269986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.271316 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.270944    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.274103 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.274085    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 19:34:31.274215 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.274196    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r5vpf\""
Jul 24 19:34:31.274542 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.274480    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 19:34:31.274640 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.274555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.276293 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.275701    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.277567 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.277345    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 19:34:31.277667 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.277645    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.277876 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.277823    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6kqlf\""
Jul 24 19:34:31.277876 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.277647    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.278893 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.278636    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 19:34:31.278893 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.278668    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 19:34:31.279617 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.279594    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.280027 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.279989    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5gzgh\""
Jul 24 19:34:31.280027 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.280020    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 19:34:31.280174 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.280061    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 19:34:31.280174 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.280082    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 19:34:31.280723 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.280528    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 19:34:31.280868 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.280748    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kkljn\""
Jul 24 19:34:31.290808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.290737    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kw6lm"
Jul 24 19:34:31.301423 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.301319    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kw6lm"
Jul 24 19:34:31.356224 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.356199    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 19:34:31.370262 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-dbus\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.370402 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-etc-kubernetes\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.370402 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370326    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7dg\" (UniqueName: \"kubernetes.io/projected/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-kube-api-access-dw7dg\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.370402 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-log-socket\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.370402 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370389    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-script-lib\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370415    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysconfig\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370439    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-conf\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370466    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370519    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-agent-certs\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-conf-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/233beb3e-a78b-411a-bdd0-20a9caf4dc22-serviceca\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370596    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cnibin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-socket-dir-parent\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.370654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370667    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-kubernetes\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-tmp\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370718    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rr9x\" (UniqueName: \"kubernetes.io/projected/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-kube-api-access-5rr9x\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370742    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk82\" (UniqueName: \"kubernetes.io/projected/086cfcea-8454-495a-8fac-bf098927b30d-kube-api-access-mxk82\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-multus\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370802    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-slash\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-var-lib-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bc36742-525e-4694-a9c0-f35221a43f28-ovn-node-metrics-cert\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cnibin\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370897    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-konnectivity-ca\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370925    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.370971    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxk65\" (UniqueName: \"kubernetes.io/projected/0bc36742-525e-4694-a9c0-f35221a43f28-kube-api-access-sxk65\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371022    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-socket-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-sys-fs\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371082    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frjt7\" (UniqueName: \"kubernetes.io/projected/d4c3522f-781e-4048-9b72-0c7577ff07f3-kube-api-access-frjt7\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.371220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-modprobe-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371127    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371154    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-k8s-cni-cncf-io\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371181    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-ovn\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-host\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371254    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086cfcea-8454-495a-8fac-bf098927b30d-tmp-dir\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371284    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-bin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/206e7d66-d68c-4d20-9821-c374c2fbade3-iptables-alerter-script\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371419    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/206e7d66-d68c-4d20-9821-c374c2fbade3-host-slash\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371446    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nh5f\" (UniqueName: \"kubernetes.io/projected/206e7d66-d68c-4d20-9821-c374c2fbade3-kube-api-access-2nh5f\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-kubelet-config\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371522    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-hostroot\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371555    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-device-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371615    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-sys\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.372162 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371682    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-multus-certs\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-config\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371762    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-systemd\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371801    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-var-lib-kubelet\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371846    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-systemd-units\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371877    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-etc-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371910    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-registration-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5wf2\" (UniqueName: \"kubernetes.io/projected/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-kube-api-access-k5wf2\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.371988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z2d7\" (UniqueName: \"kubernetes.io/projected/52a08db7-be3f-4307-a4eb-be1869fcd7b9-kube-api-access-5z2d7\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372071    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/233beb3e-a78b-411a-bdd0-20a9caf4dc22-host\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372197    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/086cfcea-8454-495a-8fac-bf098927b30d-hosts-file\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372227    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-os-release\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372258    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-netns\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372304    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-kubelet\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372328    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-run\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372352    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-lib-modules\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372403    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcx2\" (UniqueName: \"kubernetes.io/projected/233beb3e-a78b-411a-bdd0-20a9caf4dc22-kube-api-access-rbcx2\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372435    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-system-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-env-overrides\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372511    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372552    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-system-cni-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cni-binary-copy\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372602    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372642    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-tuned\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372708    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-os-release\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372758    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.372824    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.372904    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:31.872881169 +0000 UTC m=+2.967426781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-kubelet\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.373864 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.372994    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-netns\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373021    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-daemon-config\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373047    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-systemd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-node-log\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373099    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373124    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-bin\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.374690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.373165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-netd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.380905 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.380880    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:31.381024 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.380908    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:31.381024 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.380926    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:31.381148 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.381026    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:31.88100597 +0000 UTC m=+2.975551581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:31.381269 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.381248    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 19:34:31.385358 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.385274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z2d7\" (UniqueName: \"kubernetes.io/projected/52a08db7-be3f-4307-a4eb-be1869fcd7b9-kube-api-access-5z2d7\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.431066 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.430958    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal" event={"ID":"e46ae8c13e8686631be2da87086a49c5","Type":"ContainerStarted","Data":"dc8d9e4fcc92b59563726bbaec1bfec72189e7380f049a529a2259e1d4e5590f"}
Jul 24 19:34:31.432617 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.432583    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal" event={"ID":"fe59cc5e90685ef84de762ad6b19adee","Type":"ContainerStarted","Data":"9458e3428e3685aff8b90ed3f02c0f61fb8e298978a748da9d4faecdd14adb70"}
Jul 24 19:34:31.473999 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.473954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/206e7d66-d68c-4d20-9821-c374c2fbade3-iptables-alerter-script\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474006    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/206e7d66-d68c-4d20-9821-c374c2fbade3-host-slash\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nh5f\" (UniqueName: \"kubernetes.io/projected/206e7d66-d68c-4d20-9821-c374c2fbade3-kube-api-access-2nh5f\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474065    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-kubelet-config\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-hostroot\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474128    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474139    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/206e7d66-d68c-4d20-9821-c374c2fbade3-host-slash\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.474160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-device-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474179    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-sys\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474203    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474205    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-kubelet-config\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474229    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-multus-certs\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-config\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-systemd\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-var-lib-kubelet\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-systemd-units\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474430    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-etc-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-registration-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.474479    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5wf2\" (UniqueName: \"kubernetes.io/projected/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-kube-api-access-k5wf2\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474511    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/233beb3e-a78b-411a-bdd0-20a9caf4dc22-host\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474538    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/086cfcea-8454-495a-8fac-bf098927b30d-hosts-file\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.474564 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.474554    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:34:31.974539951 +0000 UTC m=+3.069085545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474578    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-os-release\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474598    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-netns\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474603    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/086cfcea-8454-495a-8fac-bf098927b30d-hosts-file\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474616    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-kubelet\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-run\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-lib-modules\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474659    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-multus-certs\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474696    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcx2\" (UniqueName: \"kubernetes.io/projected/233beb3e-a78b-411a-bdd0-20a9caf4dc22-kube-api-access-rbcx2\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/206e7d66-d68c-4d20-9821-c374c2fbade3-iptables-alerter-script\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474726    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-system-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474755    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-env-overrides\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474795    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-system-cni-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.474986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-hostroot\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-config\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475106    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-device-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475126    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-os-release\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.475296 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-netns\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475163    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-sys\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475173    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-kubelet\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475213    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-registration-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475236    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cni-binary-copy\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475258    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-systemd\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475262    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475294    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-lib-modules\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475296    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475372    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-run\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475394    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-var-lib-kubelet\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475448    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-etc-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475452    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-system-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475313    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475505    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-system-cni-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475561    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/233beb3e-a78b-411a-bdd0-20a9caf4dc22-host\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475779    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-systemd-units\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.476196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-tuned\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-os-release\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475876    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475897    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475906    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-env-overrides\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-kubelet\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475938    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-netns\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-daemon-config\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475977    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-os-release\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.475992    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-systemd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476018    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-node-log\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476044    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476070    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-bin\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-netd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476118    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-dbus\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476144    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-etc-kubernetes\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476167    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7dg\" (UniqueName: \"kubernetes.io/projected/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-kube-api-access-dw7dg\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476190    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-log-socket\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476208    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cni-binary-copy\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476216    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-script-lib\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476280    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysconfig\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-conf\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476367    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-agent-certs\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-conf-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/233beb3e-a78b-411a-bdd0-20a9caf4dc22-serviceca\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cnibin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-socket-dir-parent\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476530    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476555    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-kubernetes\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476580    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-tmp\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rr9x\" (UniqueName: \"kubernetes.io/projected/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-kube-api-access-5rr9x\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk82\" (UniqueName: \"kubernetes.io/projected/086cfcea-8454-495a-8fac-bf098927b30d-kube-api-access-mxk82\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476664    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-multus\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.477776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-slash\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-slash\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476737    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-var-lib-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bc36742-525e-4694-a9c0-f35221a43f28-ovn-node-metrics-cert\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476790    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bc36742-525e-4694-a9c0-f35221a43f28-ovnkube-script-lib\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476799    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cnibin\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-konnectivity-ca\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-socket-dir-parent\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-netns\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476854    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476876    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysconfig\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476897    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxk65\" (UniqueName: \"kubernetes.io/projected/0bc36742-525e-4694-a9c0-f35221a43f28-kube-api-access-sxk65\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477539    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-socket-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477576    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-sys-fs\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frjt7\" (UniqueName: \"kubernetes.io/projected/d4c3522f-781e-4048-9b72-0c7577ff07f3-kube-api-access-frjt7\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.478308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477637    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-modprobe-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477670    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-k8s-cni-cncf-io\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-ovn\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477759    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477790    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-host\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086cfcea-8454-495a-8fac-bf098927b30d-tmp-dir\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-bin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.477966    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-bin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.478469    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-daemon-config\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.478537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-systemd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.478595    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-node-log\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476019    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-kubelet\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.478837    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-sysctl-conf\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479086    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479170    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-bin\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.479249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479225    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-host-cni-netd\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-var-lib-cni-multus\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479371    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-dbus\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479426    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479494    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cnibin\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479559    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-kubernetes\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476554    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-log-socket\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.480046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.479984    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-socket-dir\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.480491 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480056    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-sys-fs\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.480491 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.476770    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-var-lib-openvswitch\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.480491 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480381    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-modprobe-d\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.480491 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-etc-kubernetes\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.480720 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480511    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-cni-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.480720 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480563    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-host-run-k8s-cni-cncf-io\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.480720 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc36742-525e-4694-a9c0-f35221a43f28-run-ovn\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.480720 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480702    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d4c3522f-781e-4048-9b72-0c7577ff07f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.481015 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.480993    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086cfcea-8454-495a-8fac-bf098927b30d-tmp-dir\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.481158 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.481136    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-konnectivity-ca\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.481248 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.481199    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-host\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.481315 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.481251    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-multus-conf-dir\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.481699 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.481672    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.482087 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.482062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-tmp\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.482955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.482787    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/233beb3e-a78b-411a-bdd0-20a9caf4dc22-serviceca\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.482955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.482917    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-cnibin\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.483351 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.483307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-etc-tuned\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.483351 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.483311    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bc36742-525e-4694-a9c0-f35221a43f28-ovn-node-metrics-cert\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.485655 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.485581    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb-agent-certs\") pod \"konnectivity-agent-rbnml\" (UID: \"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb\") " pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.491542 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.491519    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nh5f\" (UniqueName: \"kubernetes.io/projected/206e7d66-d68c-4d20-9821-c374c2fbade3-kube-api-access-2nh5f\") pod \"iptables-alerter-scmt7\" (UID: \"206e7d66-d68c-4d20-9821-c374c2fbade3\") " pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.493518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.493487    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcx2\" (UniqueName: \"kubernetes.io/projected/233beb3e-a78b-411a-bdd0-20a9caf4dc22-kube-api-access-rbcx2\") pod \"node-ca-jmnn5\" (UID: \"233beb3e-a78b-411a-bdd0-20a9caf4dc22\") " pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.493708 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.493685    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5wf2\" (UniqueName: \"kubernetes.io/projected/ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10-kube-api-access-k5wf2\") pod \"tuned-9f68w\" (UID: \"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10\") " pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.495404 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.495379    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frjt7\" (UniqueName: \"kubernetes.io/projected/d4c3522f-781e-4048-9b72-0c7577ff07f3-kube-api-access-frjt7\") pod \"aws-ebs-csi-driver-node-xq9vj\" (UID: \"d4c3522f-781e-4048-9b72-0c7577ff07f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.497046 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.497021    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7dg\" (UniqueName: \"kubernetes.io/projected/41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f-kube-api-access-dw7dg\") pod \"multus-ndh4r\" (UID: \"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f\") " pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.497498 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.497478    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxk65\" (UniqueName: \"kubernetes.io/projected/0bc36742-525e-4694-a9c0-f35221a43f28-kube-api-access-sxk65\") pod \"ovnkube-node-ngzdn\" (UID: \"0bc36742-525e-4694-a9c0-f35221a43f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.500912 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.500873    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk82\" (UniqueName: \"kubernetes.io/projected/086cfcea-8454-495a-8fac-bf098927b30d-kube-api-access-mxk82\") pod \"node-resolver-9dtbm\" (UID: \"086cfcea-8454-495a-8fac-bf098927b30d\") " pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.500912 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.500899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rr9x\" (UniqueName: \"kubernetes.io/projected/a12ce0e4-b9cf-4987-97a4-25fbcfccc79e-kube-api-access-5rr9x\") pod \"multus-additional-cni-plugins-nxrpz\" (UID: \"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e\") " pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.524972 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.524877    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 19:34:31.573807 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.573721    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:31.583395 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.583370    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj"
Jul 24 19:34:31.584339 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:31.584307    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc36742_525e_4694_a9c0_f35221a43f28.slice/crio-526e684d05e35bb81e4f75b99b4e4652f7ec8d8d942a3da5d6aadbc9ae4da10f WatchSource:0}: Error finding container 526e684d05e35bb81e4f75b99b4e4652f7ec8d8d942a3da5d6aadbc9ae4da10f: Status 404 returned error can't find the container with id 526e684d05e35bb81e4f75b99b4e4652f7ec8d8d942a3da5d6aadbc9ae4da10f
Jul 24 19:34:31.595927 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.595905    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9f68w"
Jul 24 19:34:31.603234 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.603206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-scmt7"
Jul 24 19:34:31.615081 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.615059    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9dtbm"
Jul 24 19:34:31.622921 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.622895    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:31.633741 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.633716    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmnn5"
Jul 24 19:34:31.641573 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.641548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxrpz"
Jul 24 19:34:31.649336 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.649318    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ndh4r"
Jul 24 19:34:31.881285 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.881174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:31.881285 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.881226    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881376    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881390    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881411    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881426    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881456    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:32.881435736 +0000 UTC m=+3.975981353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:31.881596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.881478    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:32.881467056 +0000 UTC m=+3.976012665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:31.982252 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:31.982214    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:31.982442 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.982376    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:31.982528 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:31.982449    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:34:32.98242856 +0000 UTC m=+4.076974161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:32.304489 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.304389    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 19:29:32 +0000 UTC" deadline="2028-02-18 15:10:48.805688344 +0000 UTC"
Jul 24 19:34:32.304489 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.304440    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13771h36m16.501254758s"
Jul 24 19:34:32.424667 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.424635    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:32.424871 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.424779    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:32.435003 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.434965    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"526e684d05e35bb81e4f75b99b4e4652f7ec8d8d942a3da5d6aadbc9ae4da10f"}
Jul 24 19:34:32.625864 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:32.625828    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c3522f_781e_4048_9b72_0c7577ff07f3.slice/crio-4cd95d47d4599b35873133fb079ec11fb1e245494c02f891a50a76072785e2ec WatchSource:0}: Error finding container 4cd95d47d4599b35873133fb079ec11fb1e245494c02f891a50a76072785e2ec: Status 404 returned error can't find the container with id 4cd95d47d4599b35873133fb079ec11fb1e245494c02f891a50a76072785e2ec
Jul 24 19:34:32.640297 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:32.640252    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41051d40_c8a0_4cc9_8a9a_a1a69dbbbe2f.slice/crio-03e455bfb2b256caab6b7ee6f66308e388e85c52632946dea8f0b91f08fd9e21 WatchSource:0}: Error finding container 03e455bfb2b256caab6b7ee6f66308e388e85c52632946dea8f0b91f08fd9e21: Status 404 returned error can't find the container with id 03e455bfb2b256caab6b7ee6f66308e388e85c52632946dea8f0b91f08fd9e21
Jul 24 19:34:32.642120 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:34:32.641732    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12ce0e4_b9cf_4987_97a4_25fbcfccc79e.slice/crio-0493a993f09b8390b9e0116e33fc439df8c8111bb89095da9d38b8cd14b97d57 WatchSource:0}: Error finding container 0493a993f09b8390b9e0116e33fc439df8c8111bb89095da9d38b8cd14b97d57: Status 404 returned error can't find the container with id 0493a993f09b8390b9e0116e33fc439df8c8111bb89095da9d38b8cd14b97d57
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.888746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.889005    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889134    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889198    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:34.889178448 +0000 UTC m=+5.983724067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889203    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889228    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889238    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:32.889584 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.889287    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:34.889267461 +0000 UTC m=+5.983813077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:32.990354 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:32.990290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:32.990549 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.990446    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:32.990549 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:32.990543    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:34:34.990511381 +0000 UTC m=+6.085056977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:33.305127 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.305027    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 19:29:32 +0000 UTC" deadline="2028-01-25 17:57:25.549739224 +0000 UTC"
Jul 24 19:34:33.305127 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.305072    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13198h22m52.244671492s"
Jul 24 19:34:33.427426 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.427393    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:33.427595 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:33.427542    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:33.427981 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.427957    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:33.428092 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:33.428071    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:33.455665 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.455624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9dtbm" event={"ID":"086cfcea-8454-495a-8fac-bf098927b30d","Type":"ContainerStarted","Data":"fac7fae32dafb0a3f9c4e9346353dab613cf88d8800bc4d332feda8f00bdccdc"}
Jul 24 19:34:33.463512 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.463480    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ndh4r" event={"ID":"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f","Type":"ContainerStarted","Data":"03e455bfb2b256caab6b7ee6f66308e388e85c52632946dea8f0b91f08fd9e21"}
Jul 24 19:34:33.466821 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.466745    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmnn5" event={"ID":"233beb3e-a78b-411a-bdd0-20a9caf4dc22","Type":"ContainerStarted","Data":"c09bf3f09faf255639427bd81e01d56358afd9e7ac222cb0859e95149e64bea4"}
Jul 24 19:34:33.475779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.475745    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9f68w" event={"ID":"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10","Type":"ContainerStarted","Data":"e8ea6c9a497f07e9bdf42558c1b447e5d3bb46b14c794b4acddb45830b64816e"}
Jul 24 19:34:33.479201 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.479106    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj" event={"ID":"d4c3522f-781e-4048-9b72-0c7577ff07f3","Type":"ContainerStarted","Data":"4cd95d47d4599b35873133fb079ec11fb1e245494c02f891a50a76072785e2ec"}
Jul 24 19:34:33.504545 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.503654    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal" event={"ID":"e46ae8c13e8686631be2da87086a49c5","Type":"ContainerStarted","Data":"5235a3ed85d7db0dce18c20ccb33a4e259e8ae96b7f482044ef4cc25f38b910d"}
Jul 24 19:34:33.509700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.509022    2568 generic.go:356] "Generic (PLEG): container finished" podID="fe59cc5e90685ef84de762ad6b19adee" containerID="c3f037e2be2a24ac17c93fcbe199ecaf584aafd0ed5c06392ec1d1674bafd81c" exitCode=0
Jul 24 19:34:33.509700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.509097    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal" event={"ID":"fe59cc5e90685ef84de762ad6b19adee","Type":"ContainerDied","Data":"c3f037e2be2a24ac17c93fcbe199ecaf584aafd0ed5c06392ec1d1674bafd81c"}
Jul 24 19:34:33.525223 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.525186    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerStarted","Data":"0493a993f09b8390b9e0116e33fc439df8c8111bb89095da9d38b8cd14b97d57"}
Jul 24 19:34:33.539227 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.539165    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-222.ec2.internal" podStartSLOduration=2.539144543 podStartE2EDuration="2.539144543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:33.521044227 +0000 UTC m=+4.615589842" watchObservedRunningTime="2026-07-24 19:34:33.539144543 +0000 UTC m=+4.633690164"
Jul 24 19:34:33.549868 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.549831    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-scmt7" event={"ID":"206e7d66-d68c-4d20-9821-c374c2fbade3","Type":"ContainerStarted","Data":"3e4236243370fd5714e7fe82583954462922aef9ce5988dc7063af278919388b"}
Jul 24 19:34:33.575988 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:33.571103    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbnml" event={"ID":"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb","Type":"ContainerStarted","Data":"3e29eec1aa9b9068c251e1ff6c674d3ffc822491f24925b01c84c9d39acec33d"}
Jul 24 19:34:34.424325 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:34.424288    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:34.424842 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.424458    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:34.586585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:34.586544    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal" event={"ID":"fe59cc5e90685ef84de762ad6b19adee","Type":"ContainerStarted","Data":"0646d27ea315f5ea1d45c477efa2ef2a12e6b280743e413b6a4cec29b1b5ecb9"}
Jul 24 19:34:34.605283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:34.605221    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-222.ec2.internal" podStartSLOduration=3.605197386 podStartE2EDuration="3.605197386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:34.604698264 +0000 UTC m=+5.699243881" watchObservedRunningTime="2026-07-24 19:34:34.605197386 +0000 UTC m=+5.699743004"
Jul 24 19:34:34.914404 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:34.914302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:34.914404 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:34.914368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:34.914648 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.914525    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:34.914648 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.914591    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:38.914570766 +0000 UTC m=+10.009116374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:34.915070 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.915047    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:34.915147 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.915082    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:34.915147 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.915096    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:34.915256 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:34.915148    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:38.915129988 +0000 UTC m=+10.009675595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:35.015696 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:35.015655    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:35.015888 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:35.015872    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:35.015954 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:35.015941    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:34:39.015922166 +0000 UTC m=+10.110467784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:35.427327 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:35.427297    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:35.427777 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:35.427448    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:35.428072 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:35.427848    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:35.428072 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:35.427949    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:36.425481 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:36.425293    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:36.425481 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:36.425470    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:37.424657 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:37.424587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:37.425084 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:37.424716    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:37.425084 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:37.425038    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:37.425164 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:37.425107    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:38.425411 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:38.425373    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:38.425848 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.425534    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:38.957589 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:38.957549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:38.957788 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:38.957607    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:38.957788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957753    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:38.957788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957777    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:38.958019 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957793    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:38.958019 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957806    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:38.958019 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957857    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:46.957836711 +0000 UTC m=+18.052382319 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:38.958019 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:38.957878    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:34:46.957868397 +0000 UTC m=+18.052414003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:39.059041 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:39.058994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:39.059251 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:39.059211    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:39.059353 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:39.059279    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:34:47.059260363 +0000 UTC m=+18.153805961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:39.428441 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:39.428339    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:39.428441 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:39.428339    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:39.428964 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:39.428469    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:39.428964 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:39.428628    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:40.424422 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:40.424380    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:40.424610 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:40.424544    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:41.424930 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:41.424891    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:41.425436 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:41.425031    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:41.425436 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:41.425098    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:41.425436 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:41.425200    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:42.424797 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:42.424763    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:42.424964 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:42.424886    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:43.425276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:43.425237    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:43.425684 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:43.425359    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:43.425684 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:43.425411    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:43.425684 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:43.425494    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:44.424662 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:44.424581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:44.424814 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:44.424715    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:45.424799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:45.424764    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:45.424799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:45.424781    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:45.425344 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:45.424914    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:45.425344 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:45.425036    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:46.424390 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:46.424351    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:46.424588 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:46.424466    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:47.026563 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:47.026526    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:47.027033 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:47.026571    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:47.027033 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.026722    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:34:47.027033 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.026751    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:34:47.027033 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.026764    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:47.027033 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.026823    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:03.026804334 +0000 UTC m=+34.121349926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:34:47.027325 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.026726    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:47.027325 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.027223    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:03.027207355 +0000 UTC m=+34.121752951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:34:47.127518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:47.127472    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:47.127682 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.127645    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:47.127759 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.127722    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:35:03.127701313 +0000 UTC m=+34.222246925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:34:47.424819 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:47.424741    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:47.424965 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:47.424744    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:47.424965 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.424867    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:47.424965 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:47.424952    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:48.424444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:48.424403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:48.424892 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:48.424555    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:49.425417 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:49.425381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:49.425902 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:49.425496    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:49.425902 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:49.425535    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:49.425902 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:49.425605    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:50.424270 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:50.424227    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:50.424451 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:50.424353    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:51.424697 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:51.424668    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:51.425201 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:51.424777    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:51.425201 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:51.424836    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:51.425201 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:51.424927    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:52.425121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.424815    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:52.425858 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:52.425223    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:52.664984 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.664941    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9dtbm" event={"ID":"086cfcea-8454-495a-8fac-bf098927b30d","Type":"ContainerStarted","Data":"3d4d6f192db894c05b89589d3d0863e09ec48c315fb08bb9f97089e080a83329"}
Jul 24 19:34:52.666368 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.666345    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ndh4r" event={"ID":"41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f","Type":"ContainerStarted","Data":"6849d40f74ea64fbf0239c5519b2c7160e4154e7abe09ece5be7a739a809d217"}
Jul 24 19:34:52.667548 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.667528    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmnn5" event={"ID":"233beb3e-a78b-411a-bdd0-20a9caf4dc22","Type":"ContainerStarted","Data":"5298459f6fe1f6190881bdfd993e380377531c49cc2d218a0eb9ee491194acd8"}
Jul 24 19:34:52.668870 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.668848    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9f68w" event={"ID":"ee6fba1b-bd16-460b-bbb9-a6b6aecd8e10","Type":"ContainerStarted","Data":"9bbf1851306c0fab110e206ee0843a152452a7c2a0af5a6239b9baf868dfaa19"}
Jul 24 19:34:52.670250 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.670227    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj" event={"ID":"d4c3522f-781e-4048-9b72-0c7577ff07f3","Type":"ContainerStarted","Data":"4520ff84e45e62b1ecdcc8432d677e28c204ca829179ba7e1be2a7f899c4c1ab"}
Jul 24 19:34:52.671690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.671668    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="11961053d7f83243074ab50964a9c429cdc448253eb7d694229eb8b9c0ec932d" exitCode=0
Jul 24 19:34:52.671788 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.671749    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"11961053d7f83243074ab50964a9c429cdc448253eb7d694229eb8b9c0ec932d"}
Jul 24 19:34:52.673148 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.673128    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbnml" event={"ID":"48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb","Type":"ContainerStarted","Data":"c13e23a6998dc0bc45aa147fad27e7c062707de3465f59c62242205b1a676d4c"}
Jul 24 19:34:52.675979 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.675959    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"5a57f24339e67bcfd91e158e7ecb44dfc9a540e56937ce3932b91e9bca1c4756"}
Jul 24 19:34:52.676062 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.676012    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"10797484f4755cec857592d6637abd674d08dca6616a4f1861b5461754fec910"}
Jul 24 19:34:52.676062 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.676049    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"d0f785b6890849788170733497673d54eb12ef6ad49c9729599a787b9721146b"}
Jul 24 19:34:52.676131 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.676078    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"b918583ad89bc3aad8e59a73ae57a448987844e6a9d1f8cdcadf91c170a57260"}
Jul 24 19:34:52.676131 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.676106    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"d1c341641b5b0f109938ca5fbf06b405c85711f2142ac82766b35cf3e7904672"}
Jul 24 19:34:52.676195 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.676138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"61f11ef03cc36738a9e02bc16ec689640815e92722a48a33d9f8cd50f20a79e1"}
Jul 24 19:34:52.683187 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.683147    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9dtbm" podStartSLOduration=3.770209892 podStartE2EDuration="22.683134848s" totalImagesPullingTime="18.912924956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:52.682691133 +0000 UTC m=+23.777236746" watchObservedRunningTime="2026-07-24 19:34:52.683134848 +0000 UTC m=+23.777680460"
Jul 24 19:34:52.718636 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.718593    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9f68w" podStartSLOduration=3.769854306 podStartE2EDuration="22.718580005s" totalImagesPullingTime="18.948725699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:52.69785391 +0000 UTC m=+23.792399524" watchObservedRunningTime="2026-07-24 19:34:52.718580005 +0000 UTC m=+23.813125618"
Jul 24 19:34:52.736216 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.736171    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ndh4r" podStartSLOduration=3.770210933 podStartE2EDuration="22.736156969s" totalImagesPullingTime="18.965946036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:52.736102348 +0000 UTC m=+23.830647961" watchObservedRunningTime="2026-07-24 19:34:52.736156969 +0000 UTC m=+23.830702576"
Jul 24 19:34:52.756567 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.756517    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rbnml" podStartSLOduration=3.83503796 podStartE2EDuration="22.756504669s" totalImagesPullingTime="18.921466709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:52.755947161 +0000 UTC m=+23.850492773" watchObservedRunningTime="2026-07-24 19:34:52.756504669 +0000 UTC m=+23.851050281"
Jul 24 19:34:52.769793 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.769747    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jmnn5" podStartSLOduration=3.848645843 podStartE2EDuration="22.769732957s" totalImagesPullingTime="18.921087114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:52.769599062 +0000 UTC m=+23.864144674" watchObservedRunningTime="2026-07-24 19:34:52.769732957 +0000 UTC m=+23.864278569"
Jul 24 19:34:52.979239 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:52.979217    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 19:34:53.326717 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.326495    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T19:34:52.979232721Z","UUID":"e3d87319-e7b1-4804-98ed-461161e01ae1","Handler":null,"Name":"","Endpoint":""}
Jul 24 19:34:53.328309 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.328282    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 19:34:53.328444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.328325    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 19:34:53.424735 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.424699    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:53.424921 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.424738    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:53.424921 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:53.424843    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:53.425008 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:53.424939    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:53.680784 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.680688    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-scmt7" event={"ID":"206e7d66-d68c-4d20-9821-c374c2fbade3","Type":"ContainerStarted","Data":"2f22acf9aa319d60e67ec5bb0bfcb43935c34f0ddd88362a7fa459ea66afff04"}
Jul 24 19:34:53.683367 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.683340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj" event={"ID":"d4c3522f-781e-4048-9b72-0c7577ff07f3","Type":"ContainerStarted","Data":"f3a1b5f5abba5cb652a10fccb41000b04f91a31114813c9cf4a8e250bd3a1856"}
Jul 24 19:34:53.696505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.696446    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-scmt7" podStartSLOduration=4.77817736 podStartE2EDuration="23.696427025s" totalImagesPullingTime="18.918249665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:53.695965604 +0000 UTC m=+24.790511216" watchObservedRunningTime="2026-07-24 19:34:53.696427025 +0000 UTC m=+24.790972639"
Jul 24 19:34:53.714599 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.714565    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:53.715324 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:53.715302    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:54.424571 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.424547    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:54.424733 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:54.424678    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:54.687831 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.687749    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj" event={"ID":"d4c3522f-781e-4048-9b72-0c7577ff07f3","Type":"ContainerStarted","Data":"b0bc2b1f9cb1c6b2d7ed15ef4453c166fcdee281a51e99c08a170a289f036886"}
Jul 24 19:34:54.691769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.691546    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"5b02ee8e28a7ff0d60932cc00635df9ce7f12eb7e09c5a26d0b871f6830b6e05"}
Jul 24 19:34:54.691769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.691735    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:54.692455 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.692434    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rbnml"
Jul 24 19:34:54.706989 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:54.706938    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq9vj" podStartSLOduration=3.661637791 podStartE2EDuration="24.706925122s" totalImagesPullingTime="21.045287331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:54.706464842 +0000 UTC m=+25.801010455" watchObservedRunningTime="2026-07-24 19:34:54.706925122 +0000 UTC m=+25.801470746"
Jul 24 19:34:55.328131 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:55.327944    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:34:55.410227 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:55.410192    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:34:55.425151 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:55.425121    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:55.425330 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:55.425160    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:55.425330 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:55.425240    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:55.425451 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:55.425374    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:56.424423 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:56.424392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:56.424840 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:56.424504    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:57.424452 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.424209    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:57.425120 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.424284    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:57.425120 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:57.424553    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:57.425120 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:57.424596    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:57.699283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.699151    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="53d49e63df46f567627c6d3e2e5545fb20e3efa49aedf37bd990b8cb5cd9dfc1" exitCode=0
Jul 24 19:34:57.699283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.699238    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"53d49e63df46f567627c6d3e2e5545fb20e3efa49aedf37bd990b8cb5cd9dfc1"}
Jul 24 19:34:57.702989 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.702967    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" event={"ID":"0bc36742-525e-4694-a9c0-f35221a43f28","Type":"ContainerStarted","Data":"7236ea3e454a7b7892d12ab10faabe787f0a68f713ebe1498e5b829a97826414"}
Jul 24 19:34:57.703293 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.703274    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:57.703369 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.703346    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:57.703411 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.703396    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:57.718969 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.718950    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:57.719109 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.719083    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:34:57.758056 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:57.758007    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn" podStartSLOduration=7.7176115119999995 podStartE2EDuration="27.757993074s" totalImagesPullingTime="20.040381562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:34:57.7574137 +0000 UTC m=+28.851959312" watchObservedRunningTime="2026-07-24 19:34:57.757993074 +0000 UTC m=+28.852538686"
Jul 24 19:34:58.424361 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.424330    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:58.424586 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:58.424449    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:58.706839 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.706810    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="06025c2a3852eb4f441c772e873472732a4b5b9d89f0ba69618521aa292832f6" exitCode=0
Jul 24 19:34:58.706937 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.706897    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"06025c2a3852eb4f441c772e873472732a4b5b9d89f0ba69618521aa292832f6"}
Jul 24 19:34:58.922353 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.922176    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8mr59"]
Jul 24 19:34:58.922531 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.922445    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:34:58.922596 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:58.922563    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:34:58.925004 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.924978    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pczcr"]
Jul 24 19:34:58.925129 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.925081    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:34:58.925191 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:58.925171    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:34:58.925802 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.925767    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cg6jk"]
Jul 24 19:34:58.925910 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:58.925865    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:34:58.925965 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:34:58.925949    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:34:59.136468 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:59.136391    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:34:59.193548 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:34:59.193517    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:00.424887 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.424854    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:00.425386 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.424945    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:00.425386 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:00.424952    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:35:00.425386 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.424978    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:00.425386 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:00.425050    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:35:00.425386 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:00.425112    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:35:00.714403 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.714335    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="5d9284d169d4539792aebf45cab3f3fb74f53d16d59fb266af9c36ac088c8f9c" exitCode=0
Jul 24 19:35:00.714535 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.714398    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"5d9284d169d4539792aebf45cab3f3fb74f53d16d59fb266af9c36ac088c8f9c"}
Jul 24 19:35:00.777079 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.777014    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:00.877824 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:00.877786    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:02.424586 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:02.424555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:02.425229 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:02.424604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:02.425229 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:02.424666    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:02.425229 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:02.424693    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pczcr" podUID="f06a533e-1542-4c4d-aa64-7a566d7ab0ad"
Jul 24 19:35:02.425229 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:02.424770    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cg6jk" podUID="6a44a92c-5a9b-4eab-9f5a-818c765a6920"
Jul 24 19:35:02.425229 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:02.424881    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mr59" podUID="52a08db7-be3f-4307-a4eb-be1869fcd7b9"
Jul 24 19:35:03.049505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:03.049466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:03.049505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:03.049507    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:03.049705 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049641    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 19:35:03.049705 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049667    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 19:35:03.049705 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049677    2568 projected.go:196] Error preparing data for projected volume kube-api-access-skdr2 for pod openshift-network-diagnostics/network-check-target-cg6jk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:35:03.049705 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049686    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:35:03.049823 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049726    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2 podName:6a44a92c-5a9b-4eab-9f5a-818c765a6920 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:35.049714313 +0000 UTC m=+66.144259903 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-skdr2" (UniqueName: "kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2") pod "network-check-target-cg6jk" (UID: "6a44a92c-5a9b-4eab-9f5a-818c765a6920") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 19:35:03.049823 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.049740    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs podName:52a08db7-be3f-4307-a4eb-be1869fcd7b9 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:35.049734373 +0000 UTC m=+66.144279964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs") pod "network-metrics-daemon-8mr59" (UID: "52a08db7-be3f-4307-a4eb-be1869fcd7b9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 19:35:03.150384 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:03.150339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:03.150563 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.150491    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 19:35:03.150625 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:03.150566    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret podName:f06a533e-1542-4c4d-aa64-7a566d7ab0ad nodeName:}" failed. No retries permitted until 2026-07-24 19:35:35.150545545 +0000 UTC m=+66.245091148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret") pod "global-pull-secret-syncer-pczcr" (UID: "f06a533e-1542-4c4d-aa64-7a566d7ab0ad") : object "kube-system"/"original-pull-secret" not registered
Jul 24 19:35:04.230709 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.230674    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-222.ec2.internal" event="NodeReady"
Jul 24 19:35:04.231278 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.230827    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 19:35:04.275635 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.275603    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cnzsr"]
Jul 24 19:35:04.278388 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.278362    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hp5zw"]
Jul 24 19:35:04.278529 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.278418    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.280242 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.280223    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.281156 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.281111    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8txj\""
Jul 24 19:35:04.281281 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.281111    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 19:35:04.281353 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.281322    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 19:35:04.283476 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.283454    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 19:35:04.283597 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.283539    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 19:35:04.283927 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.283898    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 19:35:04.284032 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.283970    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zq8nd\""
Jul 24 19:35:04.288687 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.288660    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cnzsr"]
Jul 24 19:35:04.292060 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.292001    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hp5zw"]
Jul 24 19:35:04.387769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.387690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:04.425373 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.425337    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:04.425553 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.425351    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:04.425553 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.425351    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:04.428420 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428396    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 19:35:04.428551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428396    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 19:35:04.428551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428513    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 19:35:04.428551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428528    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 19:35:04.428551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428544    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n5p7p\""
Jul 24 19:35:04.428819 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.428801    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-srqs8\""
Jul 24 19:35:04.457869 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.457844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5f5ca97-0438-46ca-98b3-89481ef9e461-cert\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.458004 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.457875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkw22\" (UniqueName: \"kubernetes.io/projected/d609910f-0265-420c-92db-0a74114ffb68-kube-api-access-lkw22\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.458004 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.457917    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609910f-0265-420c-92db-0a74114ffb68-tmp-dir\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.458004 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.457965    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d609910f-0265-420c-92db-0a74114ffb68-config-volume\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.458172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.458038    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.458172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.458084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr5p5\" (UniqueName: \"kubernetes.io/projected/e5f5ca97-0438-46ca-98b3-89481ef9e461-kube-api-access-wr5p5\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.489856 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.489824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:04.559094 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559055    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr5p5\" (UniqueName: \"kubernetes.io/projected/e5f5ca97-0438-46ca-98b3-89481ef9e461-kube-api-access-wr5p5\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.559298 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5f5ca97-0438-46ca-98b3-89481ef9e461-cert\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.559298 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkw22\" (UniqueName: \"kubernetes.io/projected/d609910f-0265-420c-92db-0a74114ffb68-kube-api-access-lkw22\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.559298 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609910f-0265-420c-92db-0a74114ffb68-tmp-dir\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.559298 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d609910f-0265-420c-92db-0a74114ffb68-config-volume\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.559444 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559356    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.559508 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:04.559491    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:04.559561 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:04.559551    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:05.05953418 +0000 UTC m=+36.154079790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:04.559629 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559607    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d609910f-0265-420c-92db-0a74114ffb68-tmp-dir\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.559946 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.559922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d609910f-0265-420c-92db-0a74114ffb68-config-volume\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.564152 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.564127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5f5ca97-0438-46ca-98b3-89481ef9e461-cert\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.568402 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.568344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkw22\" (UniqueName: \"kubernetes.io/projected/d609910f-0265-420c-92db-0a74114ffb68-kube-api-access-lkw22\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:04.568585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.568542    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr5p5\" (UniqueName: \"kubernetes.io/projected/e5f5ca97-0438-46ca-98b3-89481ef9e461-kube-api-access-wr5p5\") pod \"ingress-canary-hp5zw\" (UID: \"e5f5ca97-0438-46ca-98b3-89481ef9e461\") " pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.597380 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.597346    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hp5zw"
Jul 24 19:35:04.726420 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.726386    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerStarted","Data":"dd2346dc6a0781621ae8e3954a68f73ed53bf492bb7d81b76955a368c6e4941d"}
Jul 24 19:35:04.753693 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:04.753617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hp5zw"]
Jul 24 19:35:04.758307 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:04.758278    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f5ca97_0438_46ca_98b3_89481ef9e461.slice/crio-e5a6ef2b3e505b7df39c17414551fe73ecc63e4c1fc57b8e9671063aea6d1e03 WatchSource:0}: Error finding container e5a6ef2b3e505b7df39c17414551fe73ecc63e4c1fc57b8e9671063aea6d1e03: Status 404 returned error can't find the container with id e5a6ef2b3e505b7df39c17414551fe73ecc63e4c1fc57b8e9671063aea6d1e03
Jul 24 19:35:05.062470 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:05.062351    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:05.062654 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:05.062520    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:05.062654 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:05.062600    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:06.062577781 +0000 UTC m=+37.157123371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:05.730338 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:05.730295    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hp5zw" event={"ID":"e5f5ca97-0438-46ca-98b3-89481ef9e461","Type":"ContainerStarted","Data":"e5a6ef2b3e505b7df39c17414551fe73ecc63e4c1fc57b8e9671063aea6d1e03"}
Jul 24 19:35:05.733515 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:05.733481    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="dd2346dc6a0781621ae8e3954a68f73ed53bf492bb7d81b76955a368c6e4941d" exitCode=0
Jul 24 19:35:05.733651 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:05.733528    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"dd2346dc6a0781621ae8e3954a68f73ed53bf492bb7d81b76955a368c6e4941d"}
Jul 24 19:35:06.068443 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.068402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:06.068601 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.068560    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:06.068642 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.068629    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:08.068610381 +0000 UTC m=+39.163155992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:06.350683 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.350649    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jqsvf"]
Jul 24 19:35:06.355917 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.355892    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.359327 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.359304    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 19:35:06.359327 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.359321    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5rl7t\""
Jul 24 19:35:06.359532 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.359303    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 19:35:06.359532 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.359304    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 19:35:06.360760 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.360643    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 19:35:06.365086 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.365064    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jqsvf"]
Jul 24 19:35:06.366240 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.366217    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 19:35:06.450016 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.449979    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"]
Jul 24 19:35:06.456430 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.456403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"
Jul 24 19:35:06.456573 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.456463    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-22sns"]
Jul 24 19:35:06.459120 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.459096    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 24 19:35:06.459369 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.459351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4lmlp\""
Jul 24 19:35:06.459520 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.459503    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 24 19:35:06.461395 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.461379    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"]
Jul 24 19:35:06.461553 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.461533    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.464361 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.464340    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 19:35:06.464361 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.464351    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 19:35:06.464509 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.464428    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9m4tp\""
Jul 24 19:35:06.464509 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.464349    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 19:35:06.465732 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.465711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 19:35:06.466541 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.466516    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"]
Jul 24 19:35:06.466743 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.466721    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.470283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.469623    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 19:35:06.470283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.469704    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f5hn6\""
Jul 24 19:35:06.470919 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.470898    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 19:35:06.471467 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.471443    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 19:35:06.471679 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.471661    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 19:35:06.472104 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.472186 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472163    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-tmp\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.472241 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472187    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-snapshots\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.472318 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472276    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-service-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.472318 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-serving-cert\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.472417 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.472341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7v8f\" (UniqueName: \"kubernetes.io/projected/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-kube-api-access-n7v8f\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.495997 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.494453    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-656bf768f5-qbjb4"]
Jul 24 19:35:06.502280 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.498733    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.502884 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.502506    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 19:35:06.503010 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.502888    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 19:35:06.503010 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.502935    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dtjx4\""
Jul 24 19:35:06.506794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.505767    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:06.506794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.505948    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 19:35:06.506794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.506044    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 19:35:06.509799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509775    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"]
Jul 24 19:35:06.509900 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509812    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"]
Jul 24 19:35:06.509900 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509843    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-22sns"]
Jul 24 19:35:06.509900 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509862    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"]
Jul 24 19:35:06.509900 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509883    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-656bf768f5-qbjb4"]
Jul 24 19:35:06.510082 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.509984    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.512825 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.512801    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 19:35:06.513188 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.513170    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 19:35:06.513445 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.513419    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 19:35:06.513892 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.513876    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nx6vx\""
Jul 24 19:35:06.517936 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.517918    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 19:35:06.572811 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-tmp\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.572998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-snapshots\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.572998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572867    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/189ab177-f6fb-41e9-adb2-9a1340838434-serving-cert\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.572998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-service-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.572998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572937    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-trusted-ca\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.572998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-serving-cert\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573040    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7v8f\" (UniqueName: \"kubernetes.io/projected/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-kube-api-access-n7v8f\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573071    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgghr\" (UniqueName: \"kubernetes.io/projected/78f28940-98f5-4de9-b2e3-2264a18d6ca2-kube-api-access-hgghr\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573153    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-config\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.573203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573181    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.573493 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573216    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-tmp\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573493 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573232    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkjks\" (UniqueName: \"kubernetes.io/projected/189ab177-f6fb-41e9-adb2-9a1340838434-kube-api-access-qkjks\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.573493 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh2mk\" (UniqueName: \"kubernetes.io/projected/73508393-3f14-4717-98b4-a2967ce4762a-kube-api-access-xh2mk\") pod \"volume-data-source-validator-549c6d54cf-65zwf\" (UID: \"73508393-3f14-4717-98b4-a2967ce4762a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"
Jul 24 19:35:06.573587 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573507    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-service-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573587 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573565    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-snapshots\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.573951 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.573929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.577423 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.577298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-serving-cert\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.577823 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.577802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:06.584620 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.584598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7v8f\" (UniqueName: \"kubernetes.io/projected/5ba3423a-ccd8-4583-b983-fcc2e041e4aa-kube-api-access-n7v8f\") pod \"insights-operator-5885fdb759-jqsvf\" (UID: \"5ba3423a-ccd8-4583-b983-fcc2e041e4aa\") " pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.667585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.667503    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-jqsvf"
Jul 24 19:35:06.673815 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.673784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.673961 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.673825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.673961 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.673856    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxjs6\" (UniqueName: \"kubernetes.io/projected/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-kube-api-access-pxjs6\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.673961 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.673892    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkjks\" (UniqueName: \"kubernetes.io/projected/189ab177-f6fb-41e9-adb2-9a1340838434-kube-api-access-qkjks\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.673961 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.673918    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 19:35:06.674181 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.673919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674181 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.673984    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls podName:78f28940-98f5-4de9-b2e3-2264a18d6ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:07.173963843 +0000 UTC m=+38.268509434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls") pod "cluster-samples-operator-6748489656-b9rm2" (UID: "78f28940-98f5-4de9-b2e3-2264a18d6ca2") : secret "samples-operator-tls" not found
Jul 24 19:35:06.674181 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/189ab177-f6fb-41e9-adb2-9a1340838434-serving-cert\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.674181 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674103    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674181 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwkvb\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-config\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674271    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh2mk\" (UniqueName: \"kubernetes.io/projected/73508393-3f14-4717-98b4-a2967ce4762a-kube-api-access-xh2mk\") pod \"volume-data-source-validator-549c6d54cf-65zwf\" (UID: \"73508393-3f14-4717-98b4-a2967ce4762a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674335    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674366    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-trusted-ca\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674453    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674500    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.674781 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.674781 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674584    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgghr\" (UniqueName: \"kubernetes.io/projected/78f28940-98f5-4de9-b2e3-2264a18d6ca2-kube-api-access-hgghr\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.674922 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.674896    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-config\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.675152 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.675132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/189ab177-f6fb-41e9-adb2-9a1340838434-trusted-ca\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.676656 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.676637    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/189ab177-f6fb-41e9-adb2-9a1340838434-serving-cert\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.679127 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.679107    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:06.685494 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.685471    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkjks\" (UniqueName: \"kubernetes.io/projected/189ab177-f6fb-41e9-adb2-9a1340838434-kube-api-access-qkjks\") pod \"console-operator-584dcb6b54-22sns\" (UID: \"189ab177-f6fb-41e9-adb2-9a1340838434\") " pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.685662 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.685643    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh2mk\" (UniqueName: \"kubernetes.io/projected/73508393-3f14-4717-98b4-a2967ce4762a-kube-api-access-xh2mk\") pod \"volume-data-source-validator-549c6d54cf-65zwf\" (UID: \"73508393-3f14-4717-98b4-a2967ce4762a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"
Jul 24 19:35:06.685951 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.685925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgghr\" (UniqueName: \"kubernetes.io/projected/78f28940-98f5-4de9-b2e3-2264a18d6ca2-kube-api-access-hgghr\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:06.739584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.739548    2568 generic.go:356] "Generic (PLEG): container finished" podID="a12ce0e4-b9cf-4987-97a4-25fbcfccc79e" containerID="3a81272652d33715a7f74a6370087b16c5be0ef55a8df3e4ba222e3a2ee8ffa7" exitCode=0
Jul 24 19:35:06.739969 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.739601    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerDied","Data":"3a81272652d33715a7f74a6370087b16c5be0ef55a8df3e4ba222e3a2ee8ffa7"}
Jul 24 19:35:06.766652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.766615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:06.770950 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.770924    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"
Jul 24 19:35:06.775303 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775251    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.775433 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775317    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.775433 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.775433 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.775594 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.775491    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:06.775594 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775508    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.775594 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.775562    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:07.275543944 +0000 UTC m=+38.370089536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:06.775594 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775583    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxjs6\" (UniqueName: \"kubernetes.io/projected/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-kube-api-access-pxjs6\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.775612    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.775634    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:06.775688    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:07.275672389 +0000 UTC m=+38.370217979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775729    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.775799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.776048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkvb\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.776048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.776048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775891    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.776048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.775980    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.776048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.776032    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.776706 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.776684    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.778348 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.778328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.778514 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.778494    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.784975 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.784954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxjs6\" (UniqueName: \"kubernetes.io/projected/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-kube-api-access-pxjs6\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:06.785288 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.785246    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.785363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.785354    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkvb\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:06.808591 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.808552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:06.847527 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.847487    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:06.906589 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:06.906550    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:07.178416 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.178376    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:07.178580 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.178533    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 19:35:07.178624 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.178598    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls podName:78f28940-98f5-4de9-b2e3-2264a18d6ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:08.178583728 +0000 UTC m=+39.273129319 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls") pod "cluster-samples-operator-6748489656-b9rm2" (UID: "78f28940-98f5-4de9-b2e3-2264a18d6ca2") : secret "samples-operator-tls" not found
Jul 24 19:35:07.279449 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.278992    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:07.279449 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.279067    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:07.279449 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.279314    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:07.279449 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.279334    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:07.279449 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.279392    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:08.279374381 +0000 UTC m=+39.373919995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:07.280908 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.279866    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:07.280908 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:07.279927    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:08.27991141 +0000 UTC m=+39.374457003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:07.346505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.346347    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-22sns"]
Jul 24 19:35:07.348890 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.348867    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-jqsvf"]
Jul 24 19:35:07.356441 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.356419    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf"]
Jul 24 19:35:07.425730 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:07.425691    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189ab177_f6fb_41e9_adb2_9a1340838434.slice/crio-9542ea32bde89565dec515af61bd25e48f2969b2789a59771a9b9ad696c6bd64 WatchSource:0}: Error finding container 9542ea32bde89565dec515af61bd25e48f2969b2789a59771a9b9ad696c6bd64: Status 404 returned error can't find the container with id 9542ea32bde89565dec515af61bd25e48f2969b2789a59771a9b9ad696c6bd64
Jul 24 19:35:07.426116 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:07.426091    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba3423a_ccd8_4583_b983_fcc2e041e4aa.slice/crio-ab82d55f5fe8d3d11de80f7803275e0fb9d07e866a5b4c1bed46289751745aaf WatchSource:0}: Error finding container ab82d55f5fe8d3d11de80f7803275e0fb9d07e866a5b4c1bed46289751745aaf: Status 404 returned error can't find the container with id ab82d55f5fe8d3d11de80f7803275e0fb9d07e866a5b4c1bed46289751745aaf
Jul 24 19:35:07.426964 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:07.426898    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73508393_3f14_4717_98b4_a2967ce4762a.slice/crio-a4ba327874dfeb1e2cdd1f6442ffe829e8ebee8ee2e09fb0f24f47e100616934 WatchSource:0}: Error finding container a4ba327874dfeb1e2cdd1f6442ffe829e8ebee8ee2e09fb0f24f47e100616934: Status 404 returned error can't find the container with id a4ba327874dfeb1e2cdd1f6442ffe829e8ebee8ee2e09fb0f24f47e100616934
Jul 24 19:35:07.743717 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.743495    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hp5zw" event={"ID":"e5f5ca97-0438-46ca-98b3-89481ef9e461","Type":"ContainerStarted","Data":"1c90e19f586a5de633a86b9a81bce43ecead99f28005bfaa74caf798fac5d0c4"}
Jul 24 19:35:07.744628 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.744602    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" event={"ID":"189ab177-f6fb-41e9-adb2-9a1340838434","Type":"ContainerStarted","Data":"9542ea32bde89565dec515af61bd25e48f2969b2789a59771a9b9ad696c6bd64"}
Jul 24 19:35:07.745580 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.745553    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jqsvf" event={"ID":"5ba3423a-ccd8-4583-b983-fcc2e041e4aa","Type":"ContainerStarted","Data":"ab82d55f5fe8d3d11de80f7803275e0fb9d07e866a5b4c1bed46289751745aaf"}
Jul 24 19:35:07.746570 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.746550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf" event={"ID":"73508393-3f14-4717-98b4-a2967ce4762a","Type":"ContainerStarted","Data":"a4ba327874dfeb1e2cdd1f6442ffe829e8ebee8ee2e09fb0f24f47e100616934"}
Jul 24 19:35:07.749798 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.749774    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" event={"ID":"a12ce0e4-b9cf-4987-97a4-25fbcfccc79e","Type":"ContainerStarted","Data":"d3e7696870255436db703cf1b5e1d54ecca0c471bee8fde745934e01c56300f2"}
Jul 24 19:35:07.759032 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.758994    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hp5zw" podStartSLOduration=1.076584426 podStartE2EDuration="3.758981735s" totalImagesPullingTime="2.682397309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:07.758959315 +0000 UTC m=+38.853504927" watchObservedRunningTime="2026-07-24 19:35:07.758981735 +0000 UTC m=+38.853527347"
Jul 24 19:35:07.779420 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.779378    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nxrpz" podStartSLOduration=9.114543268 podStartE2EDuration="37.779362783s" totalImagesPullingTime="28.664819515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:07.778093229 +0000 UTC m=+38.872638841" watchObservedRunningTime="2026-07-24 19:35:07.779362783 +0000 UTC m=+38.873908399"
Jul 24 19:35:07.896952 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.896915    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:07.997558 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:07.997473    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:08.087095 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:08.087052    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:08.087095 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.087088    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:08.087366 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.087164    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:12.08714433 +0000 UTC m=+43.181689943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:08.102469 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:08.102437    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log" limit=1024
Jul 24 19:35:08.188204 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:08.188170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:08.188410 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.188342    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 19:35:08.188475 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.188439    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls podName:78f28940-98f5-4de9-b2e3-2264a18d6ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:10.188419605 +0000 UTC m=+41.282965223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls") pod "cluster-samples-operator-6748489656-b9rm2" (UID: "78f28940-98f5-4de9-b2e3-2264a18d6ca2") : secret "samples-operator-tls" not found
Jul 24 19:35:08.289615 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:08.289579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:08.289788 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:08.289633    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:08.289788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.289711    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:08.289788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.289766    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:08.289788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.289780    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:08.289788 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.289786    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:10.289766111 +0000 UTC m=+41.384311708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:08.290063 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:08.289818    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:10.289807887 +0000 UTC m=+41.384353478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:10.207754 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:10.207703    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:10.208195 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.207866    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 19:35:10.208195 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.207957    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls podName:78f28940-98f5-4de9-b2e3-2264a18d6ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:14.207935895 +0000 UTC m=+45.302481499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls") pod "cluster-samples-operator-6748489656-b9rm2" (UID: "78f28940-98f5-4de9-b2e3-2264a18d6ca2") : secret "samples-operator-tls" not found
Jul 24 19:35:10.309051 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:10.309009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:10.309202 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:10.309068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:10.309202 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.309171    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:10.309294 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.309201    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:10.309294 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.309212    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:10.309294 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.309242    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:14.309224176 +0000 UTC m=+45.403769789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:10.309294 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:10.309274    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:14.309251837 +0000 UTC m=+45.403797430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:11.762045 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.762016    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/0.log"
Jul 24 19:35:11.762473 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.762055    2568 generic.go:356] "Generic (PLEG): container finished" podID="189ab177-f6fb-41e9-adb2-9a1340838434" containerID="754e1d114867fcf90fa79f73ce08014ee3aac49ee16b6dd49f1d13545965ee42" exitCode=255
Jul 24 19:35:11.762473 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.762120    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" event={"ID":"189ab177-f6fb-41e9-adb2-9a1340838434","Type":"ContainerDied","Data":"754e1d114867fcf90fa79f73ce08014ee3aac49ee16b6dd49f1d13545965ee42"}
Jul 24 19:35:11.762473 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.762425    2568 scope.go:117] "RemoveContainer" containerID="754e1d114867fcf90fa79f73ce08014ee3aac49ee16b6dd49f1d13545965ee42"
Jul 24 19:35:11.763764 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.763734    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jqsvf" event={"ID":"5ba3423a-ccd8-4583-b983-fcc2e041e4aa","Type":"ContainerStarted","Data":"bae06f581fac5d1df6f186df42ef9b68051c3aa165e5deeef6bd33e3e606c0db"}
Jul 24 19:35:11.765318 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.765297    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf" event={"ID":"73508393-3f14-4717-98b4-a2967ce4762a","Type":"ContainerStarted","Data":"0f1d3eef2fe248df456feeb8b3c991ecd3d55787332faa6bb84acbe24b7f6b95"}
Jul 24 19:35:11.792395 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.792351    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-65zwf" podStartSLOduration=2.536456154 podStartE2EDuration="5.792335372s" totalImagesPullingTime="3.255879218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:11.791028666 +0000 UTC m=+42.885574281" watchObservedRunningTime="2026-07-24 19:35:11.792335372 +0000 UTC m=+42.886880984"
Jul 24 19:35:11.810366 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:11.810320    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-jqsvf" podStartSLOduration=2.524119838 podStartE2EDuration="5.810302806s" totalImagesPullingTime="3.286182968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:11.809797719 +0000 UTC m=+42.904343333" watchObservedRunningTime="2026-07-24 19:35:11.810302806 +0000 UTC m=+42.904848411"
Jul 24 19:35:12.123725 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.122887    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:12.123725 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.123104    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:12.123725 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.123167    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:20.123149711 +0000 UTC m=+51.217695314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:12.142047 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.142013    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zvz4h"]
Jul 24 19:35:12.157533 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.157495    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvz4h"]
Jul 24 19:35:12.157700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.157630    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.161111 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.161086    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 19:35:12.161271 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.161120    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mq7m6\""
Jul 24 19:35:12.161271 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.161134    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 19:35:12.223646 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.223611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0207c691-47cf-482e-bb94-7c452d36481d-data-volume\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.223794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.223662    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vwf8\" (UniqueName: \"kubernetes.io/projected/0207c691-47cf-482e-bb94-7c452d36481d-kube-api-access-4vwf8\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.223794 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.223750    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0207c691-47cf-482e-bb94-7c452d36481d-crio-socket\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.223865 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.223808    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.223865 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.223840    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324626 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0207c691-47cf-482e-bb94-7c452d36481d-data-volume\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324789 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324644    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vwf8\" (UniqueName: \"kubernetes.io/projected/0207c691-47cf-482e-bb94-7c452d36481d-kube-api-access-4vwf8\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324789 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324705    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0207c691-47cf-482e-bb94-7c452d36481d-crio-socket\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324911 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324893    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0207c691-47cf-482e-bb94-7c452d36481d-crio-socket\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324946 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324924    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.324993 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324976    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0207c691-47cf-482e-bb94-7c452d36481d-data-volume\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.325026 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.324982    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.325067 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.325049    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 19:35:12.325099 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.325067    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:12.325129 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.325124    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:12.825108445 +0000 UTC m=+43.919654036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : configmap "kube-rbac-proxy" not found
Jul 24 19:35:12.325165 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.325138    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:12.825132892 +0000 UTC m=+43.919678482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:12.337280 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.337235    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vwf8\" (UniqueName: \"kubernetes.io/projected/0207c691-47cf-482e-bb94-7c452d36481d-kube-api-access-4vwf8\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.770217 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.770193    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:35:12.770615 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.770600    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/0.log"
Jul 24 19:35:12.770653 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.770638    2568 generic.go:356] "Generic (PLEG): container finished" podID="189ab177-f6fb-41e9-adb2-9a1340838434" containerID="8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8" exitCode=255
Jul 24 19:35:12.770748 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.770726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" event={"ID":"189ab177-f6fb-41e9-adb2-9a1340838434","Type":"ContainerDied","Data":"8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8"}
Jul 24 19:35:12.770821 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.770810    2568 scope.go:117] "RemoveContainer" containerID="754e1d114867fcf90fa79f73ce08014ee3aac49ee16b6dd49f1d13545965ee42"
Jul 24 19:35:12.771118 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.771096    2568 scope.go:117] "RemoveContainer" containerID="8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8"
Jul 24 19:35:12.771341 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.771319    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-22sns_openshift-console-operator(189ab177-f6fb-41e9-adb2-9a1340838434)\"" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" podUID="189ab177-f6fb-41e9-adb2-9a1340838434"
Jul 24 19:35:12.827973 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.827936    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.828132 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.827983    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:12.828132 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.828030    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:13.828013576 +0000 UTC m=+44.922559168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:12.828132 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:12.828086    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:12.828358 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.828150    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 19:35:12.828358 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:12.828196    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:13.828186093 +0000 UTC m=+44.922731687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : configmap "kube-rbac-proxy" not found
Jul 24 19:35:13.720401 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:13.720369    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9dtbm_086cfcea-8454-495a-8fac-bf098927b30d/dns-node-resolver/0.log"
Jul 24 19:35:13.774726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:13.774698    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:35:13.775113 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:13.775011    2568 scope.go:117] "RemoveContainer" containerID="8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8"
Jul 24 19:35:13.775164 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:13.775147    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-22sns_openshift-console-operator(189ab177-f6fb-41e9-adb2-9a1340838434)\"" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" podUID="189ab177-f6fb-41e9-adb2-9a1340838434"
Jul 24 19:35:13.836102 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:13.836063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:13.836324 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:13.836127    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:13.836324 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:13.836195    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 19:35:13.836324 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:13.836213    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:13.836324 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:13.836249    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:15.836236765 +0000 UTC m=+46.930782373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : configmap "kube-rbac-proxy" not found
Jul 24 19:35:13.836324 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:13.836306    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:15.83628768 +0000 UTC m=+46.930833284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:14.240106 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:14.240063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:14.240343 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.240223    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 19:35:14.240343 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.240309    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls podName:78f28940-98f5-4de9-b2e3-2264a18d6ca2 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:22.240292457 +0000 UTC m=+53.334838052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls") pod "cluster-samples-operator-6748489656-b9rm2" (UID: "78f28940-98f5-4de9-b2e3-2264a18d6ca2") : secret "samples-operator-tls" not found
Jul 24 19:35:14.341277 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:14.341225    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:14.341459 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:14.341292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:14.341459 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.341363    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:14.341459 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.341430    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:22.341413996 +0000 UTC m=+53.435959588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:14.341459 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.341450    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:14.341638 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.341467    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:14.341638 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:14.341526    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:22.341510116 +0000 UTC m=+53.436055724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:14.720797 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:14.720769    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmnn5_233beb3e-a78b-411a-bdd0-20a9caf4dc22/node-ca/0.log"
Jul 24 19:35:15.321109 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.321083    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hp5zw_e5f5ca97-0438-46ca-98b3-89481ef9e461/serve-healthcheck-canary/0.log"
Jul 24 19:35:15.443129 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.443096    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jm9jt"]
Jul 24 19:35:15.477146 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.477110    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jm9jt"]
Jul 24 19:35:15.477321 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.477206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.480019 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.479998    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 19:35:15.480176 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.480160    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 19:35:15.480253 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.480241    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 19:35:15.480318 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.480288    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4w7wb\""
Jul 24 19:35:15.481534 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.481516    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 19:35:15.552592 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.552557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4v24\" (UniqueName: \"kubernetes.io/projected/490b7b1f-8d20-4398-b260-3c71b95e348e-kube-api-access-d4v24\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.552741 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.552628    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-cabundle\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.552741 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.552679    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-key\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.653585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.653513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4v24\" (UniqueName: \"kubernetes.io/projected/490b7b1f-8d20-4398-b260-3c71b95e348e-kube-api-access-d4v24\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.653585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.653581    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-cabundle\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.653731 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.653629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-key\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.654176 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.654148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-cabundle\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.656053 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.656023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/490b7b1f-8d20-4398-b260-3c71b95e348e-signing-key\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.661731 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.661712    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4v24\" (UniqueName: \"kubernetes.io/projected/490b7b1f-8d20-4398-b260-3c71b95e348e-kube-api-access-d4v24\") pod \"service-ca-658d7b7c78-jm9jt\" (UID: \"490b7b1f-8d20-4398-b260-3c71b95e348e\") " pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.785931 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.785904    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt"
Jul 24 19:35:15.855465 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.855434    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:15.855634 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.855487    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:15.855634 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:15.855555    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 19:35:15.855634 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:15.855601    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:19.855588326 +0000 UTC m=+50.950133917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : configmap "kube-rbac-proxy" not found
Jul 24 19:35:15.855634 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:15.855618    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:15.855858 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:15.855688    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:19.85566911 +0000 UTC m=+50.950214710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:15.918201 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:15.918133    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-jm9jt"]
Jul 24 19:35:15.920934 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:15.920912    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490b7b1f_8d20_4398_b260_3c71b95e348e.slice/crio-738c89eba979a792fc7a8f0d7073c4bee16871dcc55965d840f3334459c5de0d WatchSource:0}: Error finding container 738c89eba979a792fc7a8f0d7073c4bee16871dcc55965d840f3334459c5de0d: Status 404 returned error can't find the container with id 738c89eba979a792fc7a8f0d7073c4bee16871dcc55965d840f3334459c5de0d
Jul 24 19:35:16.784003 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:16.783969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt" event={"ID":"490b7b1f-8d20-4398-b260-3c71b95e348e","Type":"ContainerStarted","Data":"738c89eba979a792fc7a8f0d7073c4bee16871dcc55965d840f3334459c5de0d"}
Jul 24 19:35:16.809244 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:16.809207    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:16.809427 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:16.809287    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:16.809653 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:16.809641    2568 scope.go:117] "RemoveContainer" containerID="8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8"
Jul 24 19:35:16.809863 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:16.809841    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-22sns_openshift-console-operator(189ab177-f6fb-41e9-adb2-9a1340838434)\"" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" podUID="189ab177-f6fb-41e9-adb2-9a1340838434"
Jul 24 19:35:18.791116 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:18.791074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt" event={"ID":"490b7b1f-8d20-4398-b260-3c71b95e348e","Type":"ContainerStarted","Data":"a5e4e1b69f48c7329583b639844e76642a3466c38b4fdb4d61ff51633b411865"}
Jul 24 19:35:19.888765 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:19.888711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:19.889240 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:19.888783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:19.889240 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:19.888868    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 19:35:19.889240 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:19.888953    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:27.888931398 +0000 UTC m=+58.983477004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : configmap "kube-rbac-proxy" not found
Jul 24 19:35:19.889439 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:19.889293    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:19.889439 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:19.889344    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:27.889328136 +0000 UTC m=+58.983873732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:20.191649 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:20.191542    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:20.191810 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:20.191729    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 19:35:20.191810 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:20.191808    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls podName:d609910f-0265-420c-92db-0a74114ffb68 nodeName:}" failed. No retries permitted until 2026-07-24 19:35:36.19178651 +0000 UTC m=+67.286332113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls") pod "dns-default-cnzsr" (UID: "d609910f-0265-420c-92db-0a74114ffb68") : secret "dns-default-metrics-tls" not found
Jul 24 19:35:22.309318 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.309254    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:22.311778 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.311752    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f28940-98f5-4de9-b2e3-2264a18d6ca2-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-b9rm2\" (UID: \"78f28940-98f5-4de9-b2e3-2264a18d6ca2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:22.409708 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.409664    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") pod \"image-registry-656bf768f5-qbjb4\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") " pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:22.409894 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.409840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:22.409894 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:22.409847    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 19:35:22.409894 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:22.409870    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-656bf768f5-qbjb4: secret "image-registry-tls" not found
Jul 24 19:35:22.410060 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:22.409938    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls podName:10c77ec2-9999-4688-8f2d-81b05c6b486e nodeName:}" failed. No retries permitted until 2026-07-24 19:35:38.409917811 +0000 UTC m=+69.504463416 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls") pod "image-registry-656bf768f5-qbjb4" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e") : secret "image-registry-tls" not found
Jul 24 19:35:22.410060 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:22.409963    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:22.410060 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:22.410024    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls podName:3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a nodeName:}" failed. No retries permitted until 2026-07-24 19:35:38.410007511 +0000 UTC m=+69.504553105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-q9jjg" (UID: "3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a") : secret "cluster-monitoring-operator-tls" not found
Jul 24 19:35:22.416608 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.416583    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"
Jul 24 19:35:22.558245 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.558182    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-jm9jt" podStartSLOduration=5.226447642 podStartE2EDuration="7.558157626s" totalImagesPullingTime="2.331709984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:18.809135712 +0000 UTC m=+49.903681327" watchObservedRunningTime="2026-07-24 19:35:22.558157626 +0000 UTC m=+53.652703240"
Jul 24 19:35:22.559091 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.559072    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2"]
Jul 24 19:35:22.802849 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:22.802811    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2" event={"ID":"78f28940-98f5-4de9-b2e3-2264a18d6ca2","Type":"ContainerStarted","Data":"0ed4c8dfdd14d28d6aab13caa2e2ccde300a87bd83544228b9166a815f53eb16"}
Jul 24 19:35:24.811136 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:24.811058    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2" event={"ID":"78f28940-98f5-4de9-b2e3-2264a18d6ca2","Type":"ContainerStarted","Data":"4604475a2c7cce30d1a0ad87a2c9b1615c1aa0bca17589f32644d435d860b7e7"}
Jul 24 19:35:24.811136 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:24.811112    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2" event={"ID":"78f28940-98f5-4de9-b2e3-2264a18d6ca2","Type":"ContainerStarted","Data":"b0fd51789aca10e254948a3507c29d0027be3d78f14e51f718ebcac9535a8e33"}
Jul 24 19:35:24.829560 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:24.829508    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-b9rm2" podStartSLOduration=16.907516029 podStartE2EDuration="18.829490913s" totalImagesPullingTime="1.921974884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:24.828376284 +0000 UTC m=+55.922922102" watchObservedRunningTime="2026-07-24 19:35:24.829490913 +0000 UTC m=+55.924036526"
Jul 24 19:35:27.954613 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:27.954572    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:27.955089 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:27.954626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:27.955089 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:27.954774    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:27.955089 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:27.954864    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls podName:0207c691-47cf-482e-bb94-7c452d36481d nodeName:}" failed. No retries permitted until 2026-07-24 19:35:43.954843169 +0000 UTC m=+75.049388771 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvz4h" (UID: "0207c691-47cf-482e-bb94-7c452d36481d") : secret "insights-runtime-extractor-tls" not found
Jul 24 19:35:27.955249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:27.955114    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0207c691-47cf-482e-bb94-7c452d36481d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:28.424705 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:28.424633    2568 scope.go:117] "RemoveContainer" containerID="8653dcc22cfe2c11c941ce42c536b657c6f115ae284cf59d2adc47499a59f3f8"
Jul 24 19:35:28.830536 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:28.830507    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:35:28.830719 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:28.830610    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" event={"ID":"189ab177-f6fb-41e9-adb2-9a1340838434","Type":"ContainerStarted","Data":"f52780289f442e83e5c21965e13b33b8299ebc9cc6cec752ec8368c31fd6dd01"}
Jul 24 19:35:28.830906 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:28.830883    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:28.847798 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:28.847748    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-22sns" podStartSLOduration=19.565710928 podStartE2EDuration="22.847730121s" totalImagesPullingTime="3.282019193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:28.84729282 +0000 UTC m=+59.941838431" watchObservedRunningTime="2026-07-24 19:35:28.847730121 +0000 UTC m=+59.942275731"
Jul 24 19:35:29.132509 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:29.132436    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-22sns"
Jul 24 19:35:29.730881 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:29.730852    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ngzdn"
Jul 24 19:35:35.114591 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.114540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:35.114591 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.114597    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:35.117925 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.117904    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 19:35:35.117981 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.117928    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 19:35:35.127508 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.127473    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a08db7-be3f-4307-a4eb-be1869fcd7b9-metrics-certs\") pod \"network-metrics-daemon-8mr59\" (UID: \"52a08db7-be3f-4307-a4eb-be1869fcd7b9\") " pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:35.127827 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.127811    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 19:35:35.138384 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.138363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skdr2\" (UniqueName: \"kubernetes.io/projected/6a44a92c-5a9b-4eab-9f5a-818c765a6920-kube-api-access-skdr2\") pod \"network-check-target-cg6jk\" (UID: \"6a44a92c-5a9b-4eab-9f5a-818c765a6920\") " pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:35.215961 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.215910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:35.218829 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.218811    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 19:35:35.228182 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.228159    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f06a533e-1542-4c4d-aa64-7a566d7ab0ad-original-pull-secret\") pod \"global-pull-secret-syncer-pczcr\" (UID: \"f06a533e-1542-4c4d-aa64-7a566d7ab0ad\") " pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:35.337109 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.337072    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pczcr"
Jul 24 19:35:35.347854 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.347826    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n5p7p\""
Jul 24 19:35:35.353650 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.353630    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-srqs8\""
Jul 24 19:35:35.355745 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.355722    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:35.361778 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.361753    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mr59"
Jul 24 19:35:35.485323 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.485293    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pczcr"]
Jul 24 19:35:35.488042 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:35.488013    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06a533e_1542_4c4d_aa64_7a566d7ab0ad.slice/crio-9fb8cd9429b34ca8cdc2a85f9fb7544cf708415c07c25a511a94d2bb731b6b9a WatchSource:0}: Error finding container 9fb8cd9429b34ca8cdc2a85f9fb7544cf708415c07c25a511a94d2bb731b6b9a: Status 404 returned error can't find the container with id 9fb8cd9429b34ca8cdc2a85f9fb7544cf708415c07c25a511a94d2bb731b6b9a
Jul 24 19:35:35.496221 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.496194    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cg6jk"]
Jul 24 19:35:35.498495 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:35.498469    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a44a92c_5a9b_4eab_9f5a_818c765a6920.slice/crio-1b6815cb919e9e7a088453cf2581e285c0a45d953e89567211bb2e6c7f46e378 WatchSource:0}: Error finding container 1b6815cb919e9e7a088453cf2581e285c0a45d953e89567211bb2e6c7f46e378: Status 404 returned error can't find the container with id 1b6815cb919e9e7a088453cf2581e285c0a45d953e89567211bb2e6c7f46e378
Jul 24 19:35:35.513951 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.513926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8mr59"]
Jul 24 19:35:35.516728 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:35.516704    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a08db7_be3f_4307_a4eb_be1869fcd7b9.slice/crio-a45c5d9d225bd4b8ceb28991c99880ca966697e227672101454cf29a7f5a52c8 WatchSource:0}: Error finding container a45c5d9d225bd4b8ceb28991c99880ca966697e227672101454cf29a7f5a52c8: Status 404 returned error can't find the container with id a45c5d9d225bd4b8ceb28991c99880ca966697e227672101454cf29a7f5a52c8
Jul 24 19:35:35.852190 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.852157    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mr59" event={"ID":"52a08db7-be3f-4307-a4eb-be1869fcd7b9","Type":"ContainerStarted","Data":"a45c5d9d225bd4b8ceb28991c99880ca966697e227672101454cf29a7f5a52c8"}
Jul 24 19:35:35.853319 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.853285    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cg6jk" event={"ID":"6a44a92c-5a9b-4eab-9f5a-818c765a6920","Type":"ContainerStarted","Data":"1b6815cb919e9e7a088453cf2581e285c0a45d953e89567211bb2e6c7f46e378"}
Jul 24 19:35:35.854441 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:35.854421    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pczcr" event={"ID":"f06a533e-1542-4c4d-aa64-7a566d7ab0ad","Type":"ContainerStarted","Data":"9fb8cd9429b34ca8cdc2a85f9fb7544cf708415c07c25a511a94d2bb731b6b9a"}
Jul 24 19:35:36.225733 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.225492    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:36.236953 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.234080    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d609910f-0265-420c-92db-0a74114ffb68-metrics-tls\") pod \"dns-default-cnzsr\" (UID: \"d609910f-0265-420c-92db-0a74114ffb68\") " pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:36.387115 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.386733    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-656bf768f5-qbjb4"]
Jul 24 19:35:36.387115 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:35:36.387053    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-656bf768f5-qbjb4" podUID="10c77ec2-9999-4688-8f2d-81b05c6b486e"
Jul 24 19:35:36.390863 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.390585    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-s2l2l"]
Jul 24 19:35:36.394616 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.394395    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8txj\""
Jul 24 19:35:36.402524 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.402494    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:36.415070 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.409942    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"]
Jul 24 19:35:36.415070 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.411389    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:36.415070 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.414556    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 19:35:36.415070 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.414869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-blfbw\""
Jul 24 19:35:36.418751 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.418727    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 19:35:36.438099 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.437854    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"]
Jul 24 19:35:36.438270 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.438169    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.441740 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.441310    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 19:35:36.442382 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.442357    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-55t92\""
Jul 24 19:35:36.442890 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.442861    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 19:35:36.443135 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.443113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 19:35:36.443234 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.443180    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 19:35:36.453944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.453777    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"]
Jul 24 19:35:36.453944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.453815    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-s2l2l"]
Jul 24 19:35:36.453944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.453854    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"]
Jul 24 19:35:36.453944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.453875    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bf74b7f4b-sw28q"]
Jul 24 19:35:36.453944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.453946    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.458411 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.458388    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 19:35:36.458620 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.458602    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 19:35:36.458707 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.458627    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 19:35:36.458937 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.458919    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 19:35:36.467943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.467729    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf74b7f4b-sw28q"]
Jul 24 19:35:36.467943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.467851    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.527603 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bbb91b-77b3-424d-bcbe-1d48595da1a8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527643    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7260b00e-cf5d-48ae-b928-ca2b9901b97a-ca-trust-extracted\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccxx9\" (UniqueName: \"kubernetes.io/projected/66bbb91b-77b3-424d-bcbe-1d48595da1a8-kube-api-access-ccxx9\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-trusted-ca\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.527779 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk2xt\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-kube-api-access-gk2xt\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527819    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527871    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxxqw\" (UniqueName: \"kubernetes.io/projected/2d980ff5-4ea0-4aa5-a12c-b9c444e7b976-kube-api-access-kxxqw\") pod \"downloads-7c97dc4555-s2l2l\" (UID: \"2d980ff5-4ea0-4aa5-a12c-b9c444e7b976\") " pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527900    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ea41836-682f-4a0e-9265-1f003c9942cd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjtzn\" (UniqueName: \"kubernetes.io/projected/4ea41836-682f-4a0e-9265-1f003c9942cd-kube-api-access-fjtzn\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527962    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-installation-pull-secrets\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.527993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-bound-sa-token\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.528023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-certificates\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.528058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-image-registry-private-configuration\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.528121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.528099    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-tls\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.628856 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.628814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.628884    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxxqw\" (UniqueName: \"kubernetes.io/projected/2d980ff5-4ea0-4aa5-a12c-b9c444e7b976-kube-api-access-kxxqw\") pod \"downloads-7c97dc4555-s2l2l\" (UID: \"2d980ff5-4ea0-4aa5-a12c-b9c444e7b976\") " pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:36.629050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.628913    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ea41836-682f-4a0e-9265-1f003c9942cd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.629050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.628941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjtzn\" (UniqueName: \"kubernetes.io/projected/4ea41836-682f-4a0e-9265-1f003c9942cd-kube-api-access-fjtzn\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.629050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.628975    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-installation-pull-secrets\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629026    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-bound-sa-token\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629057    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-certificates\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629088    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-image-registry-private-configuration\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-tls\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629162    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bbb91b-77b3-424d-bcbe-1d48595da1a8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629202    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629228    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629272    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7260b00e-cf5d-48ae-b928-ca2b9901b97a-ca-trust-extracted\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629300    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccxx9\" (UniqueName: \"kubernetes.io/projected/66bbb91b-77b3-424d-bcbe-1d48595da1a8-kube-api-access-ccxx9\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629852 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.629852 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629387    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-trusted-ca\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.629852 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.629413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk2xt\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-kube-api-access-gk2xt\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.631395 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.631367    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bbb91b-77b3-424d-bcbe-1d48595da1a8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.631516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.631391    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-certificates\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.632356 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.632100    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7260b00e-cf5d-48ae-b928-ca2b9901b97a-ca-trust-extracted\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.632988 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.632953    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7260b00e-cf5d-48ae-b928-ca2b9901b97a-trusted-ca\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.635279 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.635232    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-image-registry-private-configuration\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.635826 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.635432    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.635826 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.635787    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7260b00e-cf5d-48ae-b928-ca2b9901b97a-installation-pull-secrets\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.636042 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.635999    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4ea41836-682f-4a0e-9265-1f003c9942cd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.636817 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.636734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-registry-tls\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.637201 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.637162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-hub\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.637819 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.637774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.638196 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.638175    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/66bbb91b-77b3-424d-bcbe-1d48595da1a8-ca\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.640437 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.640354    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk2xt\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-kube-api-access-gk2xt\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.642014 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.641955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjtzn\" (UniqueName: \"kubernetes.io/projected/4ea41836-682f-4a0e-9265-1f003c9942cd-kube-api-access-fjtzn\") pod \"managed-serviceaccount-addon-agent-985cb57b4-qxw26\" (UID: \"4ea41836-682f-4a0e-9265-1f003c9942cd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.651276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.651206    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccxx9\" (UniqueName: \"kubernetes.io/projected/66bbb91b-77b3-424d-bcbe-1d48595da1a8-kube-api-access-ccxx9\") pod \"cluster-proxy-proxy-agent-6bb659fccb-bk6r8\" (UID: \"66bbb91b-77b3-424d-bcbe-1d48595da1a8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.652273 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.652205    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxxqw\" (UniqueName: \"kubernetes.io/projected/2d980ff5-4ea0-4aa5-a12c-b9c444e7b976-kube-api-access-kxxqw\") pod \"downloads-7c97dc4555-s2l2l\" (UID: \"2d980ff5-4ea0-4aa5-a12c-b9c444e7b976\") " pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:36.653470 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.653425    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7260b00e-cf5d-48ae-b928-ca2b9901b97a-bound-sa-token\") pod \"image-registry-bf74b7f4b-sw28q\" (UID: \"7260b00e-cf5d-48ae-b928-ca2b9901b97a\") " pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.729893 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.729855    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:36.760625 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.760547    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"
Jul 24 19:35:36.775726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.775690    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"
Jul 24 19:35:36.793950 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.793734    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nx6vx\""
Jul 24 19:35:36.802071 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.801959    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:36.861131 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.860644    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:36.900286 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.897678    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:36.993123 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:36.993090    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cnzsr"]
Jul 24 19:35:37.018292 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.016726    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-s2l2l"]
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033068    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033128    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033167    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033200    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwkvb\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033279    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033369    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.033395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates\") pod \"10c77ec2-9999-4688-8f2d-81b05c6b486e\" (UID: \"10c77ec2-9999-4688-8f2d-81b05c6b486e\") "
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.034152    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.034541    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:35:37.036700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.034852    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:35:37.043070 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.043010    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:35:37.043526 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.043488    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:35:37.044666 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.044622    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:35:37.046193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.045885    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8"]
Jul 24 19:35:37.046458 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.046420    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb" (OuterVolumeSpecName: "kube-api-access-zwkvb") pod "10c77ec2-9999-4688-8f2d-81b05c6b486e" (UID: "10c77ec2-9999-4688-8f2d-81b05c6b486e"). InnerVolumeSpecName "kube-api-access-zwkvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:35:37.064405 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.064374    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26"]
Jul 24 19:35:37.088589 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.088560    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf74b7f4b-sw28q"]
Jul 24 19:35:37.129402 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:37.129361    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd609910f_0265_420c_92db_0a74114ffb68.slice/crio-e1c0f7d6cb836ef470204ad9dde205f26665a1f7e0738696c76ecbe759aa4789 WatchSource:0}: Error finding container e1c0f7d6cb836ef470204ad9dde205f26665a1f7e0738696c76ecbe759aa4789: Status 404 returned error can't find the container with id e1c0f7d6cb836ef470204ad9dde205f26665a1f7e0738696c76ecbe759aa4789
Jul 24 19:35:37.134185 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:37.133754    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66bbb91b_77b3_424d_bcbe_1d48595da1a8.slice/crio-6e9ddb42b680450a24079e29d590cea49d737db1135e6e4e1b0dea0e5b650329 WatchSource:0}: Error finding container 6e9ddb42b680450a24079e29d590cea49d737db1135e6e4e1b0dea0e5b650329: Status 404 returned error can't find the container with id 6e9ddb42b680450a24079e29d590cea49d737db1135e6e4e1b0dea0e5b650329
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:37.134561    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ea41836_682f_4a0e_9265_1f003c9942cd.slice/crio-ef2cb4ca23aac85ac100df1949b0daa56092e0dada2efce4b703132dcb2e8a7c WatchSource:0}: Error finding container ef2cb4ca23aac85ac100df1949b0daa56092e0dada2efce4b703132dcb2e8a7c: Status 404 returned error can't find the container with id ef2cb4ca23aac85ac100df1949b0daa56092e0dada2efce4b703132dcb2e8a7c
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134574    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-installation-pull-secrets\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134597    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c77ec2-9999-4688-8f2d-81b05c6b486e-ca-trust-extracted\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134612    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-bound-sa-token\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134626    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwkvb\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-kube-api-access-zwkvb\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134640    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c77ec2-9999-4688-8f2d-81b05c6b486e-image-registry-private-configuration\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134655    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-trusted-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.134977 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.134670    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-certificates\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:37.135711 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:37.135687    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7260b00e_cf5d_48ae_b928_ca2b9901b97a.slice/crio-83ba060baf04e1351e2cc92e4963c5b525fd6b9db2baf9452f749c1accb58b3d WatchSource:0}: Error finding container 83ba060baf04e1351e2cc92e4963c5b525fd6b9db2baf9452f749c1accb58b3d: Status 404 returned error can't find the container with id 83ba060baf04e1351e2cc92e4963c5b525fd6b9db2baf9452f749c1accb58b3d
Jul 24 19:35:37.870451 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.870021    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q" event={"ID":"7260b00e-cf5d-48ae-b928-ca2b9901b97a","Type":"ContainerStarted","Data":"40b2f6418506cf478ad20b48d3ff9267cd996b72fa23b9aa3fe264ce1d5d9650"}
Jul 24 19:35:37.870451 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.870148    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q" event={"ID":"7260b00e-cf5d-48ae-b928-ca2b9901b97a","Type":"ContainerStarted","Data":"83ba060baf04e1351e2cc92e4963c5b525fd6b9db2baf9452f749c1accb58b3d"}
Jul 24 19:35:37.870451 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.870306    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:37.872683 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.872632    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8" event={"ID":"66bbb91b-77b3-424d-bcbe-1d48595da1a8","Type":"ContainerStarted","Data":"6e9ddb42b680450a24079e29d590cea49d737db1135e6e4e1b0dea0e5b650329"}
Jul 24 19:35:37.874557 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.874517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26" event={"ID":"4ea41836-682f-4a0e-9265-1f003c9942cd","Type":"ContainerStarted","Data":"ef2cb4ca23aac85ac100df1949b0daa56092e0dada2efce4b703132dcb2e8a7c"}
Jul 24 19:35:37.879718 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.879659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-s2l2l" event={"ID":"2d980ff5-4ea0-4aa5-a12c-b9c444e7b976","Type":"ContainerStarted","Data":"39990a95e3391d76f03cb4ae361358933155f5c26b5642340f76894661a325c4"}
Jul 24 19:35:37.881312 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.881286    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cnzsr" event={"ID":"d609910f-0265-420c-92db-0a74114ffb68","Type":"ContainerStarted","Data":"e1c0f7d6cb836ef470204ad9dde205f26665a1f7e0738696c76ecbe759aa4789"}
Jul 24 19:35:37.883443 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.883409    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-656bf768f5-qbjb4"
Jul 24 19:35:37.884365 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.884343    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mr59" event={"ID":"52a08db7-be3f-4307-a4eb-be1869fcd7b9","Type":"ContainerStarted","Data":"d60c4581617cebf2fd770784b3a6994f80d45a597d26602ac5ae79da86f7775d"}
Jul 24 19:35:37.884471 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.884390    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mr59" event={"ID":"52a08db7-be3f-4307-a4eb-be1869fcd7b9","Type":"ContainerStarted","Data":"2a58b61a33817b1aa4e09e81869cf8f5910eb403e7bf3174664d4dafe2917c70"}
Jul 24 19:35:37.888686 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.888637    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q" podStartSLOduration=1.888620697 podStartE2EDuration="1.888620697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:37.887824007 +0000 UTC m=+68.982369620" watchObservedRunningTime="2026-07-24 19:35:37.888620697 +0000 UTC m=+68.983166310"
Jul 24 19:35:37.906625 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.906569    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8mr59" podStartSLOduration=66.250210402 podStartE2EDuration="1m7.906548016s" totalImagesPullingTime="1.656337614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:37.90604455 +0000 UTC m=+69.000590164" watchObservedRunningTime="2026-07-24 19:35:37.906548016 +0000 UTC m=+69.001093630"
Jul 24 19:35:37.933541 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.933488    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-656bf768f5-qbjb4"]
Jul 24 19:35:37.936813 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:37.936754    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-656bf768f5-qbjb4"]
Jul 24 19:35:38.052623 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:38.052566    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c77ec2-9999-4688-8f2d-81b05c6b486e-registry-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:35:38.457679 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:38.457068    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:38.473692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:38.473517    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-q9jjg\" (UID: \"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:38.625603 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:38.625351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dtjx4\""
Jul 24 19:35:38.633040 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:38.632950    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"
Jul 24 19:35:39.431557 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:39.431516    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c77ec2-9999-4688-8f2d-81b05c6b486e" path="/var/lib/kubelet/pods/10c77ec2-9999-4688-8f2d-81b05c6b486e/volumes"
Jul 24 19:35:44.013803 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:44.013762    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:44.016471 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:44.016441    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0207c691-47cf-482e-bb94-7c452d36481d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvz4h\" (UID: \"0207c691-47cf-482e-bb94-7c452d36481d\") " pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:44.269364 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:44.269281    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mq7m6\""
Jul 24 19:35:44.277772 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:44.277739    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvz4h"
Jul 24 19:35:44.962573 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:44.962498    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg"]
Jul 24 19:35:45.259469 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:35:45.259433    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba1fbf3_0dc3_4aa6_be2f_ef5ed4fb052a.slice/crio-d3327b2fb734f29ff245bdbf599ecca34c218ae9ddc960653cbe97316cc7c873 WatchSource:0}: Error finding container d3327b2fb734f29ff245bdbf599ecca34c218ae9ddc960653cbe97316cc7c873: Status 404 returned error can't find the container with id d3327b2fb734f29ff245bdbf599ecca34c218ae9ddc960653cbe97316cc7c873
Jul 24 19:35:45.462356 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.462275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvz4h"]
Jul 24 19:35:45.923613 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.923519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvz4h" event={"ID":"0207c691-47cf-482e-bb94-7c452d36481d","Type":"ContainerStarted","Data":"146253f3b94fd9ef765822c3393622d4bc990e89b30304cd425cdf475eddbb17"}
Jul 24 19:35:45.926513 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.926480    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8" event={"ID":"66bbb91b-77b3-424d-bcbe-1d48595da1a8","Type":"ContainerStarted","Data":"c416d9ce918131015e6874299d857b0da16964606bcf5bcabbe0c2a27e23b34c"}
Jul 24 19:35:45.926653 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.926530    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8" event={"ID":"66bbb91b-77b3-424d-bcbe-1d48595da1a8","Type":"ContainerStarted","Data":"8a32a982de2f54f7334cec7c56962e99a37ab2c1cd25fe6d0ce95d5e82af7e53"}
Jul 24 19:35:45.926653 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.926581    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8" event={"ID":"66bbb91b-77b3-424d-bcbe-1d48595da1a8","Type":"ContainerStarted","Data":"6888b5eb3c6b38df4d510181fc0c4c01addd4d6007e50ff0585b99237703b905"}
Jul 24 19:35:45.929173 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.929146    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26" event={"ID":"4ea41836-682f-4a0e-9265-1f003c9942cd","Type":"ContainerStarted","Data":"07bb22789a401de94a5499fe9d1b8a937b3172da7663cc69347dfdd8f39dee7a"}
Jul 24 19:35:45.932756 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.932705    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg" event={"ID":"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a","Type":"ContainerStarted","Data":"d3327b2fb734f29ff245bdbf599ecca34c218ae9ddc960653cbe97316cc7c873"}
Jul 24 19:35:45.934998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.934956    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cg6jk" event={"ID":"6a44a92c-5a9b-4eab-9f5a-818c765a6920","Type":"ContainerStarted","Data":"f1faeaa161fece7c5bf8bb815cd96c99bf800a50548248598e1c573b29ddce0f"}
Jul 24 19:35:45.935313 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.935291    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:35:45.937643 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.937582    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pczcr" event={"ID":"f06a533e-1542-4c4d-aa64-7a566d7ab0ad","Type":"ContainerStarted","Data":"9698be5629feda7b0a7b13c3c55cda3ec61d8db5eba2b9afb19c0df447c6a3ac"}
Jul 24 19:35:45.940220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.939685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cnzsr" event={"ID":"d609910f-0265-420c-92db-0a74114ffb68","Type":"ContainerStarted","Data":"40488002f75555145ce16b3d4abc2e753a8f123cd4a3215ba81575677f4f73ca"}
Jul 24 19:35:45.940220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.939728    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cnzsr" event={"ID":"d609910f-0265-420c-92db-0a74114ffb68","Type":"ContainerStarted","Data":"b5f60e0241e605bf2ecc2576e45ffe62095aa6c08cb1b9ca5c93751fb1ff84a2"}
Jul 24 19:35:45.940220 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.940173    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:45.945838 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.945698    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bb659fccb-bk6r8" podStartSLOduration=1.80854004 podStartE2EDuration="9.945681494s" totalImagesPullingTime="8.137141454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:45.945061665 +0000 UTC m=+77.039607279" watchObservedRunningTime="2026-07-24 19:35:45.945681494 +0000 UTC m=+77.040227104"
Jul 24 19:35:45.982976 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.982456    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pczcr" podStartSLOduration=66.185424966 podStartE2EDuration="1m15.982435385s" totalImagesPullingTime="9.797010419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:45.98218168 +0000 UTC m=+77.076727295" watchObservedRunningTime="2026-07-24 19:35:45.982435385 +0000 UTC m=+77.076981008"
Jul 24 19:35:45.983518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.983148    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-985cb57b4-qxw26" podStartSLOduration=1.853449992 podStartE2EDuration="9.983137862s" totalImagesPullingTime="8.12968787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:45.965546314 +0000 UTC m=+77.060091930" watchObservedRunningTime="2026-07-24 19:35:45.983137862 +0000 UTC m=+77.077683471"
Jul 24 19:35:46.000230 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:45.999445    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cnzsr" podStartSLOduration=33.850563088 podStartE2EDuration="41.999425853s" totalImagesPullingTime="8.148862765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:45.99878742 +0000 UTC m=+77.093333034" watchObservedRunningTime="2026-07-24 19:35:45.999425853 +0000 UTC m=+77.093971466"
Jul 24 19:35:46.019686 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:46.018061    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cg6jk" podStartSLOduration=66.278829667 podStartE2EDuration="1m16.018041952s" totalImagesPullingTime="9.739212285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:46.017698732 +0000 UTC m=+77.112244346" watchObservedRunningTime="2026-07-24 19:35:46.018041952 +0000 UTC m=+77.112587566"
Jul 24 19:35:47.954517 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:47.954372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg" event={"ID":"3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a","Type":"ContainerStarted","Data":"ca1e2e63de256629aa6a4a45cfdb8bbffe7e749d25244311f4b852d4b3261c92"}
Jul 24 19:35:47.972689 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:47.972636    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-q9jjg" podStartSLOduration=39.510760749 podStartE2EDuration="41.972620815s" totalImagesPullingTime="2.461860066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:47.97168682 +0000 UTC m=+79.066232436" watchObservedRunningTime="2026-07-24 19:35:47.972620815 +0000 UTC m=+79.067166482"
Jul 24 19:35:56.958542 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:56.958411    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cnzsr"
Jul 24 19:35:58.897315 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:58.897285    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bf74b7f4b-sw28q"
Jul 24 19:35:59.001515 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:59.001470    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-s2l2l" event={"ID":"2d980ff5-4ea0-4aa5-a12c-b9c444e7b976","Type":"ContainerStarted","Data":"f204d0186498448471b5298f131a7a39b750ce29fb83415f021ac9364c93a38f"}
Jul 24 19:35:59.001831 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:59.001781    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:35:59.003228 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:59.003200    2568 patch_prober.go:28] interesting pod/downloads-7c97dc4555-s2l2l container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.16:8080/\": dial tcp 10.133.0.16:8080: connect: connection refused" start-of-body=
Jul 24 19:35:59.003349 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:59.003251    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-s2l2l" podUID="2d980ff5-4ea0-4aa5-a12c-b9c444e7b976" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.16:8080/\": dial tcp 10.133.0.16:8080: connect: connection refused"
Jul 24 19:35:59.019949 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:35:59.019908    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-s2l2l" podStartSLOduration=1.51674855 podStartE2EDuration="23.019892085s" totalImagesPullingTime="21.503143535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:35:59.018176049 +0000 UTC m=+90.112721663" watchObservedRunningTime="2026-07-24 19:35:59.019892085 +0000 UTC m=+90.114437698"
Jul 24 19:36:00.010769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.010718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvz4h" event={"ID":"0207c691-47cf-482e-bb94-7c452d36481d","Type":"ContainerStarted","Data":"b11ce17088eda65d0d77bbd41e5a04d7680212368eeb9cb4aea7d0f727ad0e62"}
Jul 24 19:36:00.021787 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.021755    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-s2l2l"
Jul 24 19:36:00.761811 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.759717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b9mfm"]
Jul 24 19:36:00.792071 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.792031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.795798 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.795768    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 19:36:00.796329 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.796047    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 19:36:00.796943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.796485    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z4drf\""
Jul 24 19:36:00.798580 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.797216    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 19:36:00.803938 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.798899    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 19:36:00.856226 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856181    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-root\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856243    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-textfile\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856301    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-metrics-client-ca\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856349    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzg96\" (UniqueName: \"kubernetes.io/projected/a69f0e14-d134-4c35-8ab8-b730dde3913d-kube-api-access-qzg96\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856379    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-wtmp\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856406    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856669 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856439    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856669 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856488    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-sys\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.856669 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.856521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.956933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-wtmp\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.956980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-sys\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-root\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-textfile\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957185    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-metrics-client-ca\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957234    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzg96\" (UniqueName: \"kubernetes.io/projected/a69f0e14-d134-4c35-8ab8-b730dde3913d-kube-api-access-qzg96\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.957490    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-wtmp\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.958135    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958276 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:36:00.958242    2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 19:36:00.958955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.958521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-root\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.958955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.958575    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a69f0e14-d134-4c35-8ab8-b730dde3913d-sys\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.961275 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.959149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-textfile\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.961275 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:36:00.959309    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls podName:a69f0e14-d134-4c35-8ab8-b730dde3913d nodeName:}" failed. No retries permitted until 2026-07-24 19:36:01.459288912 +0000 UTC m=+92.553834503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls") pod "node-exporter-b9mfm" (UID: "a69f0e14-d134-4c35-8ab8-b730dde3913d") : secret "node-exporter-tls" not found
Jul 24 19:36:00.961275 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.959506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a69f0e14-d134-4c35-8ab8-b730dde3913d-metrics-client-ca\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.961465 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.961344    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:00.971551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:00.967849    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzg96\" (UniqueName: \"kubernetes.io/projected/a69f0e14-d134-4c35-8ab8-b730dde3913d-kube-api-access-qzg96\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:01.461362 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:01.461298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:01.464170 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:01.464146    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a69f0e14-d134-4c35-8ab8-b730dde3913d-node-exporter-tls\") pod \"node-exporter-b9mfm\" (UID: \"a69f0e14-d134-4c35-8ab8-b730dde3913d\") " pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:01.710246 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:01.710030    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9mfm"
Jul 24 19:36:04.119159 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:36:04.119114    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69f0e14_d134_4c35_8ab8_b730dde3913d.slice/crio-b4de21b283d5ed54a4d54e7838cf42362b707da0b4d24888d59b20e6437e0004 WatchSource:0}: Error finding container b4de21b283d5ed54a4d54e7838cf42362b707da0b4d24888d59b20e6437e0004: Status 404 returned error can't find the container with id b4de21b283d5ed54a4d54e7838cf42362b707da0b4d24888d59b20e6437e0004
Jul 24 19:36:05.043406 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:05.043366    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvz4h" event={"ID":"0207c691-47cf-482e-bb94-7c452d36481d","Type":"ContainerStarted","Data":"a9cb3e51f92fcee3b89fab70547b606809b48ad2a5d5edb0742d9b81814d382c"}
Jul 24 19:36:05.045130 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:05.045097    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9mfm" event={"ID":"a69f0e14-d134-4c35-8ab8-b730dde3913d","Type":"ContainerStarted","Data":"b4de21b283d5ed54a4d54e7838cf42362b707da0b4d24888d59b20e6437e0004"}
Jul 24 19:36:06.053375 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:06.053336    2568 generic.go:356] "Generic (PLEG): container finished" podID="a69f0e14-d134-4c35-8ab8-b730dde3913d" containerID="953f8cca30e7e729877b07e447178883e514fc88c21fae5c06c5bbe07357a6fe" exitCode=0
Jul 24 19:36:06.053933 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:06.053392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9mfm" event={"ID":"a69f0e14-d134-4c35-8ab8-b730dde3913d","Type":"ContainerDied","Data":"953f8cca30e7e729877b07e447178883e514fc88c21fae5c06c5bbe07357a6fe"}
Jul 24 19:36:07.062410 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:07.062366    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9mfm" event={"ID":"a69f0e14-d134-4c35-8ab8-b730dde3913d","Type":"ContainerStarted","Data":"9a44faa56a3e89b3d6dd55a58413dd5d9c61328fd1c873b11e1df9ba097fc74f"}
Jul 24 19:36:07.062982 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:07.062439    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9mfm" event={"ID":"a69f0e14-d134-4c35-8ab8-b730dde3913d","Type":"ContainerStarted","Data":"da66b6ca31af8dae1ffe5aaace91324431763fcb7d901984b6097e6759646a59"}
Jul 24 19:36:07.083512 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:07.083373    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b9mfm" podStartSLOduration=5.533591735 podStartE2EDuration="7.08335499s" totalImagesPullingTime="1.449660598s" totalInitContainerRuntime="100.102657ms" isStatefulPod=true podCreationTimestamp="2026-07-24 19:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:36:07.082785415 +0000 UTC m=+98.177331030" watchObservedRunningTime="2026-07-24 19:36:07.08335499 +0000 UTC m=+98.177900602"
Jul 24 19:36:10.076728 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:10.076687    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvz4h" event={"ID":"0207c691-47cf-482e-bb94-7c452d36481d","Type":"ContainerStarted","Data":"bc34cd1742f237d277ed3f16f566a351d8419a2ec64f91c2d784e4cd8717ceb9"}
Jul 24 19:36:10.100802 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:10.100753    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zvz4h" podStartSLOduration=34.140314441 podStartE2EDuration="58.100737908s" totalImagesPullingTime="23.960423467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 19:35:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:36:10.09982884 +0000 UTC m=+101.194374459" watchObservedRunningTime="2026-07-24 19:36:10.100737908 +0000 UTC m=+101.195283562"
Jul 24 19:36:16.949161 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:16.949129    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cg6jk"
Jul 24 19:36:22.120119 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:22.120079    2568 generic.go:356] "Generic (PLEG): container finished" podID="5ba3423a-ccd8-4583-b983-fcc2e041e4aa" containerID="bae06f581fac5d1df6f186df42ef9b68051c3aa165e5deeef6bd33e3e606c0db" exitCode=0
Jul 24 19:36:22.120702 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:22.120161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jqsvf" event={"ID":"5ba3423a-ccd8-4583-b983-fcc2e041e4aa","Type":"ContainerDied","Data":"bae06f581fac5d1df6f186df42ef9b68051c3aa165e5deeef6bd33e3e606c0db"}
Jul 24 19:36:22.120702 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:22.120621    2568 scope.go:117] "RemoveContainer" containerID="bae06f581fac5d1df6f186df42ef9b68051c3aa165e5deeef6bd33e3e606c0db"
Jul 24 19:36:23.125516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:23.125479    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-jqsvf" event={"ID":"5ba3423a-ccd8-4583-b983-fcc2e041e4aa","Type":"ContainerStarted","Data":"59cb0d4b73aaa89b9888f01a3cc2c5aa33f9e6a47e1ca8769ffa3953d4e63818"}
Jul 24 19:36:25.321622 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:25.321586    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-q9jjg_3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a/cluster-monitoring-operator/0.log"
Jul 24 19:36:26.518710 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:26.518678    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/init-textfile/0.log"
Jul 24 19:36:26.720821 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:26.720795    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/node-exporter/0.log"
Jul 24 19:36:26.918950 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:26.918867    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/kube-rbac-proxy/0.log"
Jul 24 19:36:32.320522 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:32.320485    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:36:32.522201 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:32.522161    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/2.log"
Jul 24 19:36:33.121340 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:33.121300    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-s2l2l_2d980ff5-4ea0-4aa5-a12c-b9c444e7b976/download-server/0.log"
Jul 24 19:36:33.519214 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:36:33.519177    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hp5zw_e5f5ca97-0438-46ca-98b3-89481ef9e461/serve-healthcheck-canary/0.log"
Jul 24 19:39:29.262051 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:39:29.262023    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:39:29.270382 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:39:29.270356    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:39:29.275146 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:39:29.275124    2568 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 19:40:03.043166 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.043131    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"]
Jul 24 19:40:03.045427 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.045409    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.048239 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.048214    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 19:40:03.048336 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.048315    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 19:40:03.048635 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.048621    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 19:40:03.048954 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.048928    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2ws7w\""
Jul 24 19:40:03.048954 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.048942    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 19:40:03.063050 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.063030    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"]
Jul 24 19:40:03.142448 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.142406    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-webhook-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.142652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.142457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-manifests\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.142652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.142585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7zx7\" (UniqueName: \"kubernetes.io/projected/97897029-4b76-40d5-b09b-679295507f40-kube-api-access-s7zx7\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.142652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.142622    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.142790 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.142687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-tmp\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.243995 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.243954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-tmp\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.244179 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.244006    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-webhook-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.244179 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.244045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-manifests\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.244179 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.244106    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zx7\" (UniqueName: \"kubernetes.io/projected/97897029-4b76-40d5-b09b-679295507f40-kube-api-access-s7zx7\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.244179 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.244158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.244499 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.244479    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-manifests\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.246275 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.246238    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97897029-4b76-40d5-b09b-679295507f40-tmp\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.246465 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.246445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-webhook-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.246522 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.246506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/97897029-4b76-40d5-b09b-679295507f40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.254838 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.254818    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7zx7\" (UniqueName: \"kubernetes.io/projected/97897029-4b76-40d5-b09b-679295507f40-kube-api-access-s7zx7\") pod \"opendatahub-operator-controller-manager-d7d8f86c4-6n4tp\" (UID: \"97897029-4b76-40d5-b09b-679295507f40\") " pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.354859 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.354772    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:03.492464 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.492426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"]
Jul 24 19:40:03.496130 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:03.496099    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97897029_4b76_40d5_b09b_679295507f40.slice/crio-d05e4bf2a377aa891da6fa1765981caa941ffffd105331d17fd4e4b49f6806f2 WatchSource:0}: Error finding container d05e4bf2a377aa891da6fa1765981caa941ffffd105331d17fd4e4b49f6806f2: Status 404 returned error can't find the container with id d05e4bf2a377aa891da6fa1765981caa941ffffd105331d17fd4e4b49f6806f2
Jul 24 19:40:03.497919 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.497899    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:40:03.898525 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:03.898487    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp" event={"ID":"97897029-4b76-40d5-b09b-679295507f40","Type":"ContainerStarted","Data":"d05e4bf2a377aa891da6fa1765981caa941ffffd105331d17fd4e4b49f6806f2"}
Jul 24 19:40:06.914742 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:06.914656    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp" event={"ID":"97897029-4b76-40d5-b09b-679295507f40","Type":"ContainerStarted","Data":"9ea9585576701669efeb39b77b886f8d85eaab882fe4e33faf6582cbaa05a524"}
Jul 24 19:40:07.919774 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:07.919740    2568 generic.go:356] "Generic (PLEG): container finished" podID="97897029-4b76-40d5-b09b-679295507f40" containerID="9ea9585576701669efeb39b77b886f8d85eaab882fe4e33faf6582cbaa05a524" exitCode=0
Jul 24 19:40:07.920148 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:07.919783    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp" event={"ID":"97897029-4b76-40d5-b09b-679295507f40","Type":"ContainerDied","Data":"9ea9585576701669efeb39b77b886f8d85eaab882fe4e33faf6582cbaa05a524"}
Jul 24 19:40:08.925518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:08.925487    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp" event={"ID":"97897029-4b76-40d5-b09b-679295507f40","Type":"ContainerStarted","Data":"98ab700e98dc7bf8d2d27d8d3d129d16f79e1d56edabcf0da593c95e89571187"}
Jul 24 19:40:08.925882 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:08.925636    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:08.949041 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:08.948971    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp" podStartSLOduration=1.444382569 podStartE2EDuration="5.948956918s" totalImagesPullingTime="3.432637417s" totalInitContainerRuntime="1.071936932s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:08.946712765 +0000 UTC m=+340.041258421" watchObservedRunningTime="2026-07-24 19:40:08.948956918 +0000 UTC m=+340.043502530"
Jul 24 19:40:19.932017 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:19.931976    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d7d8f86c4-6n4tp"
Jul 24 19:40:28.991560 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:28.991471    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"]
Jul 24 19:40:28.994660 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:28.994628    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:28.999563 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:28.999538    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 19:40:29.000391 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.000370    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 19:40:29.001328 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.000889    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-86bd7\""
Jul 24 19:40:29.001328 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.000934    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 19:40:29.001328 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.001158    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 19:40:29.006391 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.006371    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"]
Jul 24 19:40:29.068504 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.068468    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jbbt8"]
Jul 24 19:40:29.071583 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.071561    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.074192 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.074170    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 19:40:29.074319 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.074196    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-446f4\""
Jul 24 19:40:29.082732 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.082708    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jbbt8"]
Jul 24 19:40:29.154522 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.154488    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tls-certs\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.154695 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.154527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.154695 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.154557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhcr4\" (UniqueName: \"kubernetes.io/projected/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-kube-api-access-bhcr4\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.154695 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.154640    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tmp\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.154695 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.154671    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68kp\" (UniqueName: \"kubernetes.io/projected/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-kube-api-access-h68kp\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.255672 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.255593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tmp\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.255672 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.255634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68kp\" (UniqueName: \"kubernetes.io/projected/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-kube-api-access-h68kp\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.255898 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.255681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tls-certs\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.255898 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.255697    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.255898 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.255716    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcr4\" (UniqueName: \"kubernetes.io/projected/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-kube-api-access-bhcr4\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.264275 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.259187    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tmp\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.266298 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.266247    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 19:40:29.266546 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.266523    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 19:40:29.266889 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:29.266870    2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 24 19:40:29.266958 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:29.266935    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert podName:86d1b4ca-93ac-44e0-b322-18f5bad6ffc4 nodeName:}" failed. No retries permitted until 2026-07-24 19:40:29.766915852 +0000 UTC m=+360.861461460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert") pod "odh-model-controller-7998c67dc4-jbbt8" (UID: "86d1b4ca-93ac-44e0-b322-18f5bad6ffc4") : secret "odh-model-controller-webhook-cert" not found
Jul 24 19:40:29.269322 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.269127    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 19:40:29.269941 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.269893    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-tls-certs\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.270252 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.270224    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68kp\" (UniqueName: \"kubernetes.io/projected/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-kube-api-access-h68kp\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.280283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.279735    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 19:40:29.289519 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.289491    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcr4\" (UniqueName: \"kubernetes.io/projected/27ce4ae1-2cc4-4471-9eb5-1eef532168b0-kube-api-access-bhcr4\") pod \"kube-auth-proxy-85c49bfc89-66vph\" (UID: \"27ce4ae1-2cc4-4471-9eb5-1eef532168b0\") " pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.306877 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.306847    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-86bd7\""
Jul 24 19:40:29.315038 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.315015    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"
Jul 24 19:40:29.466616 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.466586    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85c49bfc89-66vph"]
Jul 24 19:40:29.468824 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:29.468790    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ce4ae1_2cc4_4471_9eb5_1eef532168b0.slice/crio-4ad5611bdfaed6a35229b953aeb6dd08fe939df97b2427fa410d2ff0fa13bc46 WatchSource:0}: Error finding container 4ad5611bdfaed6a35229b953aeb6dd08fe939df97b2427fa410d2ff0fa13bc46: Status 404 returned error can't find the container with id 4ad5611bdfaed6a35229b953aeb6dd08fe939df97b2427fa410d2ff0fa13bc46
Jul 24 19:40:29.733103 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.733064    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-px656"]
Jul 24 19:40:29.737393 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.737372    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.740067 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.740046    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 19:40:29.748801 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.748773    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-px656"]
Jul 24 19:40:29.862160 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.862117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.862363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.862173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbjzb\" (UniqueName: \"kubernetes.io/projected/e7a783e4-94d5-4d79-988f-1a2cad8ab410-kube-api-access-kbjzb\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.862363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.862204    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a783e4-94d5-4d79-988f-1a2cad8ab410-tls-certs\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.864526 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.864492    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d1b4ca-93ac-44e0-b322-18f5bad6ffc4-cert\") pod \"odh-model-controller-7998c67dc4-jbbt8\" (UID: \"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4\") " pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:29.963729 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.963690    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbjzb\" (UniqueName: \"kubernetes.io/projected/e7a783e4-94d5-4d79-988f-1a2cad8ab410-kube-api-access-kbjzb\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.963729 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.963744    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a783e4-94d5-4d79-988f-1a2cad8ab410-tls-certs\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.966194 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.966169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a783e4-94d5-4d79-988f-1a2cad8ab410-tls-certs\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.972145 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.972112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbjzb\" (UniqueName: \"kubernetes.io/projected/e7a783e4-94d5-4d79-988f-1a2cad8ab410-kube-api-access-kbjzb\") pod \"model-serving-api-54d89bd5fc-px656\" (UID: \"e7a783e4-94d5-4d79-988f-1a2cad8ab410\") " pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:29.984443 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.984368    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-446f4\""
Jul 24 19:40:29.992363 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:29.992336    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:30.002673 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.002640    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph" event={"ID":"27ce4ae1-2cc4-4471-9eb5-1eef532168b0","Type":"ContainerStarted","Data":"4ad5611bdfaed6a35229b953aeb6dd08fe939df97b2427fa410d2ff0fa13bc46"}
Jul 24 19:40:30.047620 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.047581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:30.098192 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.098104    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:30.103532 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.103472    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.104893 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.104246    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:30.108308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.107355    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 19:40:30.108308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.107580    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 19:40:30.108308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.107822    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 19:40:30.108308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.108001    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 19:40:30.108308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.108190    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c9qrb\""
Jul 24 19:40:30.147491 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.147459    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jbbt8"]
Jul 24 19:40:30.151492 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:30.151454    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d1b4ca_93ac_44e0_b322_18f5bad6ffc4.slice/crio-1bbbdec2ca77ca7edb74c2972ccc5445ce5d40b28a4bf1cbd04e354d97b5dacd WatchSource:0}: Error finding container 1bbbdec2ca77ca7edb74c2972ccc5445ce5d40b28a4bf1cbd04e354d97b5dacd: Status 404 returned error can't find the container with id 1bbbdec2ca77ca7edb74c2972ccc5445ce5d40b28a4bf1cbd04e354d97b5dacd
Jul 24 19:40:30.213535 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.213502    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-px656"]
Jul 24 19:40:30.218996 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:30.218968    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a783e4_94d5_4d79_988f_1a2cad8ab410.slice/crio-ce3a46755a0a1b835b7f8d6c950a0d1acaf3adbff36d8f341f785b5f9a0ac8c4 WatchSource:0}: Error finding container ce3a46755a0a1b835b7f8d6c950a0d1acaf3adbff36d8f341f785b5f9a0ac8c4: Status 404 returned error can't find the container with id ce3a46755a0a1b835b7f8d6c950a0d1acaf3adbff36d8f341f785b5f9a0ac8c4
Jul 24 19:40:30.267117 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.266931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267117 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.266979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267117 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.267087    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnt2d\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267436 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.267143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267436 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.267183    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267436 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.267208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.267436 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.267294    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368401 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnt2d\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368585 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368554    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368613    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.368799 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.368640    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.369404 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.369375    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.371889 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.371844    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.372167 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.372144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.372242 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.372220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.372687 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.372664    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.376574 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.376544    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.376994 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.376798    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnt2d\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d\") pod \"istiod-openshift-gateway-7b7995d5cd-25b8x\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.418868 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.418837    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:30.576312 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:30.576277    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:30.580173 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:30.580142    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988ab5ad_2b07_43e0_a316_098813e21679.slice/crio-c3473f4bc08a0821418687671999147a774136e0237511292ee5f87ec22af38e WatchSource:0}: Error finding container c3473f4bc08a0821418687671999147a774136e0237511292ee5f87ec22af38e: Status 404 returned error can't find the container with id c3473f4bc08a0821418687671999147a774136e0237511292ee5f87ec22af38e
Jul 24 19:40:31.011773 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:31.011735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-px656" event={"ID":"e7a783e4-94d5-4d79-988f-1a2cad8ab410","Type":"ContainerStarted","Data":"ce3a46755a0a1b835b7f8d6c950a0d1acaf3adbff36d8f341f785b5f9a0ac8c4"}
Jul 24 19:40:31.013236 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:31.013203    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8" event={"ID":"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4","Type":"ContainerStarted","Data":"1bbbdec2ca77ca7edb74c2972ccc5445ce5d40b28a4bf1cbd04e354d97b5dacd"}
Jul 24 19:40:31.014566 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:31.014540    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" event={"ID":"988ab5ad-2b07-43e0-a316-098813e21679","Type":"ContainerStarted","Data":"c3473f4bc08a0821418687671999147a774136e0237511292ee5f87ec22af38e"}
Jul 24 19:40:35.430104 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:35.430065    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 19:40:35.436764 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:35.436729    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 19:40:35.436923 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:35.436795    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 19:40:36.044138 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.044094    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8" event={"ID":"86d1b4ca-93ac-44e0-b322-18f5bad6ffc4","Type":"ContainerStarted","Data":"512af1dba5b0c251360dace8f2265012d35f8ca3d5f5710ea9a52fa923e21a10"}
Jul 24 19:40:36.044366 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.044185    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:36.045671 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.045643    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph" event={"ID":"27ce4ae1-2cc4-4471-9eb5-1eef532168b0","Type":"ContainerStarted","Data":"3467c29b3967553e40cdabc5e000c48f3610531d81650053ed84c228954fcb19"}
Jul 24 19:40:36.047047 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.047022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" event={"ID":"988ab5ad-2b07-43e0-a316-098813e21679","Type":"ContainerStarted","Data":"9d5a07146943ab498276cafb0ef89c8807fac731154f2033c71166d0d004cdb2"}
Jul 24 19:40:36.047177 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.047071    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:36.048583 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.048561    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-px656" event={"ID":"e7a783e4-94d5-4d79-988f-1a2cad8ab410","Type":"ContainerStarted","Data":"a3c36389f69ace701d5a34f0f7a6b2e9011b1ed8ed3dcf76da5e3deb5d974506"}
Jul 24 19:40:36.048700 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.048686    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:40:36.049210 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.049190    2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-25b8x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 19:40:36.049314 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.049229    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" podUID="988ab5ad-2b07-43e0-a316-098813e21679" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 19:40:36.061660 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.061602    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8" podStartSLOduration=1.771822212 podStartE2EDuration="7.06158495s" totalImagesPullingTime="5.289762738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:36.059752501 +0000 UTC m=+367.154298115" watchObservedRunningTime="2026-07-24 19:40:36.06158495 +0000 UTC m=+367.156130565"
Jul 24 19:40:36.076923 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.076849    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-px656" podStartSLOduration=1.871211723 podStartE2EDuration="7.076829699s" totalImagesPullingTime="5.205617976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:36.07526969 +0000 UTC m=+367.169815320" watchObservedRunningTime="2026-07-24 19:40:36.076829699 +0000 UTC m=+367.171375317"
Jul 24 19:40:36.095394 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.095348    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" podStartSLOduration=1.241316136 podStartE2EDuration="6.095331987s" totalImagesPullingTime="4.854015851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:36.093417504 +0000 UTC m=+367.187963119" watchObservedRunningTime="2026-07-24 19:40:36.095331987 +0000 UTC m=+367.189877600"
Jul 24 19:40:36.108542 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:36.108498    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85c49bfc89-66vph" podStartSLOduration=2.152108411 podStartE2EDuration="8.108483985s" totalImagesPullingTime="5.956375574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:36.107843163 +0000 UTC m=+367.202388776" watchObservedRunningTime="2026-07-24 19:40:36.108483985 +0000 UTC m=+367.203029598"
Jul 24 19:40:37.053838 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:37.053811    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:40.278738 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.278704    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"]
Jul 24 19:40:40.282161 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.282144    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.287713 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.287691    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 19:40:40.288817 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.288796    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xm5wd\""
Jul 24 19:40:40.306811 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.306789    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"]
Jul 24 19:40:40.321087 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.321060    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"]
Jul 24 19:40:40.324167 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.324147    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.335727 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.335708    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-f57gk\""
Jul 24 19:40:40.337952 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.337934    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 19:40:40.347053 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.347031    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"]
Jul 24 19:40:40.358208 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.358166    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.358346 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.358213    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bb4w\" (UniqueName: \"kubernetes.io/projected/f05f5dd3-3210-416e-ae8f-844ba4e45080-kube-api-access-7bb4w\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.358417 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.358394    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.358460 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.358447    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mv6t\" (UniqueName: \"kubernetes.io/projected/ddcd5f0d-269b-4420-af46-8e88c2329ee8-kube-api-access-5mv6t\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.459769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.459723    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.459934 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.459781    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mv6t\" (UniqueName: \"kubernetes.io/projected/ddcd5f0d-269b-4420-af46-8e88c2329ee8-kube-api-access-5mv6t\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.459934 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.459828    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.459934 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:40.459920    2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 24 19:40:40.460127 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.459947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bb4w\" (UniqueName: \"kubernetes.io/projected/f05f5dd3-3210-416e-ae8f-844ba4e45080-kube-api-access-7bb4w\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.460127 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:40.459977    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert podName:f05f5dd3-3210-416e-ae8f-844ba4e45080 nodeName:}" failed. No retries permitted until 2026-07-24 19:40:40.959958281 +0000 UTC m=+372.054503875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert") pod "kserve-controller-manager-d6f4b65b8-w4jz8" (UID: "f05f5dd3-3210-416e-ae8f-844ba4e45080") : secret "kserve-webhook-server-cert" not found
Jul 24 19:40:40.460127 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:40.459923    2568 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 19:40:40.460127 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:40:40.460062    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert podName:ddcd5f0d-269b-4420-af46-8e88c2329ee8 nodeName:}" failed. No retries permitted until 2026-07-24 19:40:40.960053442 +0000 UTC m=+372.054599040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert") pod "llmisvc-controller-manager-777c6bd7d5-zbmdv" (UID: "ddcd5f0d-269b-4420-af46-8e88c2329ee8") : secret "llmisvc-webhook-server-cert" not found
Jul 24 19:40:40.471167 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.471130    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bb4w\" (UniqueName: \"kubernetes.io/projected/f05f5dd3-3210-416e-ae8f-844ba4e45080-kube-api-access-7bb4w\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.471308 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.471198    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mv6t\" (UniqueName: \"kubernetes.io/projected/ddcd5f0d-269b-4420-af46-8e88c2329ee8-kube-api-access-5mv6t\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.964071 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.964017    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:40.964249 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.964113    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.966788 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.966762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f05f5dd3-3210-416e-ae8f-844ba4e45080-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4jz8\" (UID: \"f05f5dd3-3210-416e-ae8f-844ba4e45080\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:40.966859 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:40.966762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddcd5f0d-269b-4420-af46-8e88c2329ee8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zbmdv\" (UID: \"ddcd5f0d-269b-4420-af46-8e88c2329ee8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:41.193002 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:41.192959    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:41.235685 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:41.235604    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:41.337380 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:41.337348    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"]
Jul 24 19:40:41.342081 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:41.342054    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05f5dd3_3210_416e_ae8f_844ba4e45080.slice/crio-b8d41245d865623ba60bbf0dac3d31dc2f01dc457a2b56ac206c17021e84c3b0 WatchSource:0}: Error finding container b8d41245d865623ba60bbf0dac3d31dc2f01dc457a2b56ac206c17021e84c3b0: Status 404 returned error can't find the container with id b8d41245d865623ba60bbf0dac3d31dc2f01dc457a2b56ac206c17021e84c3b0
Jul 24 19:40:41.382688 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:41.382663    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"]
Jul 24 19:40:41.384559 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:40:41.384535    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podddcd5f0d_269b_4420_af46_8e88c2329ee8.slice/crio-fc39820c962839efd2dec05009d517b62c3e5339e88a76fb8bd4a66a61fdbd6e WatchSource:0}: Error finding container fc39820c962839efd2dec05009d517b62c3e5339e88a76fb8bd4a66a61fdbd6e: Status 404 returned error can't find the container with id fc39820c962839efd2dec05009d517b62c3e5339e88a76fb8bd4a66a61fdbd6e
Jul 24 19:40:42.076382 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:42.076341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv" event={"ID":"ddcd5f0d-269b-4420-af46-8e88c2329ee8","Type":"ContainerStarted","Data":"fc39820c962839efd2dec05009d517b62c3e5339e88a76fb8bd4a66a61fdbd6e"}
Jul 24 19:40:42.077499 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:42.077469    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8" event={"ID":"f05f5dd3-3210-416e-ae8f-844ba4e45080","Type":"ContainerStarted","Data":"b8d41245d865623ba60bbf0dac3d31dc2f01dc457a2b56ac206c17021e84c3b0"}
Jul 24 19:40:43.397125 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:43.397076    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:43.397594 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:43.397359    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" podUID="988ab5ad-2b07-43e0-a316-098813e21679" containerName="discovery" containerID="cri-o://9d5a07146943ab498276cafb0ef89c8807fac731154f2033c71166d0d004cdb2" gracePeriod=30
Jul 24 19:40:44.090097 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.090054    2568 generic.go:356] "Generic (PLEG): container finished" podID="988ab5ad-2b07-43e0-a316-098813e21679" containerID="9d5a07146943ab498276cafb0ef89c8807fac731154f2033c71166d0d004cdb2" exitCode=0
Jul 24 19:40:44.090253 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.090100    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" event={"ID":"988ab5ad-2b07-43e0-a316-098813e21679","Type":"ContainerDied","Data":"9d5a07146943ab498276cafb0ef89c8807fac731154f2033c71166d0d004cdb2"}
Jul 24 19:40:44.507084 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.507061    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:44.597437 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597400    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597592 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597592 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597531    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnt2d\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597592 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597582    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597751 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597608    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597751 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597638    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.597751 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.597695    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert\") pod \"988ab5ad-2b07-43e0-a316-098813e21679\" (UID: \"988ab5ad-2b07-43e0-a316-098813e21679\") "
Jul 24 19:40:44.598339 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.598295    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:40:44.600440 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600404    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts" (OuterVolumeSpecName: "cacerts") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:40:44.600440 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600403    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d" (OuterVolumeSpecName: "kube-api-access-mnt2d") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "kube-api-access-mnt2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:40:44.600619 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600475    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:40:44.600619 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600555    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:40:44.600619 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600575    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs" (OuterVolumeSpecName: "local-certs") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:40:44.600847 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.600826    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token" (OuterVolumeSpecName: "istio-token") pod "988ab5ad-2b07-43e0-a316-098813e21679" (UID: "988ab5ad-2b07-43e0-a316-098813e21679"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:40:44.698606 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698576    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-dns-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698606 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698606    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-istio-token\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698606 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698615    2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/988ab5ad-2b07-43e0-a316-098813e21679-local-certs\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698834 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698623    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnt2d\" (UniqueName: \"kubernetes.io/projected/988ab5ad-2b07-43e0-a316-098813e21679-kube-api-access-mnt2d\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698834 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698631    2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-cacerts\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698834 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698641    2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/988ab5ad-2b07-43e0-a316-098813e21679-istio-kubeconfig\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:44.698834 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:44.698650    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/988ab5ad-2b07-43e0-a316-098813e21679-istio-csr-ca-configmap\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:40:45.095435 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.095398    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv" event={"ID":"ddcd5f0d-269b-4420-af46-8e88c2329ee8","Type":"ContainerStarted","Data":"3d3ecbedf071c7da037af25c7ebb7b584f64ff13edea1380cd73bc089c5f859b"}
Jul 24 19:40:45.095631 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.095505    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:40:45.096902 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.096881    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"
Jul 24 19:40:45.096902 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.096889    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x" event={"ID":"988ab5ad-2b07-43e0-a316-098813e21679","Type":"ContainerDied","Data":"c3473f4bc08a0821418687671999147a774136e0237511292ee5f87ec22af38e"}
Jul 24 19:40:45.097230 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.096925    2568 scope.go:117] "RemoveContainer" containerID="9d5a07146943ab498276cafb0ef89c8807fac731154f2033c71166d0d004cdb2"
Jul 24 19:40:45.098466 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.098441    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8" event={"ID":"f05f5dd3-3210-416e-ae8f-844ba4e45080","Type":"ContainerStarted","Data":"49d8429056d2d78aa5a22570e00047af9cdae6b96e61002556f220758ec2de09"}
Jul 24 19:40:45.098605 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.098587    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:40:45.114936 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.114897    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv" podStartSLOduration=1.9548967830000001 podStartE2EDuration="5.114883931s" totalImagesPullingTime="3.159987148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:45.113771259 +0000 UTC m=+376.208316872" watchObservedRunningTime="2026-07-24 19:40:45.114883931 +0000 UTC m=+376.209429561"
Jul 24 19:40:45.129459 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.129434    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:45.134659 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.134632    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-25b8x"]
Jul 24 19:40:45.149354 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.149306    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8" podStartSLOduration=1.996227882 podStartE2EDuration="5.14929148s" totalImagesPullingTime="3.153063598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:40:45.148730535 +0000 UTC m=+376.243276147" watchObservedRunningTime="2026-07-24 19:40:45.14929148 +0000 UTC m=+376.243837093"
Jul 24 19:40:45.429619 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:45.429582    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988ab5ad-2b07-43e0-a316-098813e21679" path="/var/lib/kubelet/pods/988ab5ad-2b07-43e0-a316-098813e21679/volumes"
Jul 24 19:40:47.056043 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:47.056012    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jbbt8"
Jul 24 19:40:47.058342 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:40:47.058324    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-px656"
Jul 24 19:41:16.106428 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:16.106390    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zbmdv"
Jul 24 19:41:16.109434 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:16.109417    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4jz8"
Jul 24 19:41:24.681584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.681549    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:24.681942 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.681840    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988ab5ad-2b07-43e0-a316-098813e21679" containerName="discovery"
Jul 24 19:41:24.681942 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.681851    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="988ab5ad-2b07-43e0-a316-098813e21679" containerName="discovery"
Jul 24 19:41:24.681942 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.681916    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="988ab5ad-2b07-43e0-a316-098813e21679" containerName="discovery"
Jul 24 19:41:24.684789 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.684769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:24.689063 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.689042    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 19:41:24.689195 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.689098    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 19:41:24.689195 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.689173    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-n7ctd\""
Jul 24 19:41:24.692845 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.692782    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:24.836616 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.836579    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq5fk\" (UniqueName: \"kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk\") pod \"kuadrant-operator-catalog-jb46b\" (UID: \"7c05249b-866f-4fd6-aa58-82e9c0f08d59\") " pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:24.937254 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.937170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq5fk\" (UniqueName: \"kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk\") pod \"kuadrant-operator-catalog-jb46b\" (UID: \"7c05249b-866f-4fd6-aa58-82e9c0f08d59\") " pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:24.945593 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.945562    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq5fk\" (UniqueName: \"kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk\") pod \"kuadrant-operator-catalog-jb46b\" (UID: \"7c05249b-866f-4fd6-aa58-82e9c0f08d59\") " pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:24.995530 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:24.995492    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:25.040752 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.040720    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:25.123948 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.123921    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:25.126311 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:41:25.126286    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c05249b_866f_4fd6_aa58_82e9c0f08d59.slice/crio-c08c08d73f1213cee4efbcba1cb763bd928bedc370a6618a014da76faeb239f8 WatchSource:0}: Error finding container c08c08d73f1213cee4efbcba1cb763bd928bedc370a6618a014da76faeb239f8: Status 404 returned error can't find the container with id c08c08d73f1213cee4efbcba1cb763bd928bedc370a6618a014da76faeb239f8
Jul 24 19:41:25.249752 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.249717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jp5r8"]
Jul 24 19:41:25.254029 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.254011    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:25.259942 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.259915    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jp5r8"]
Jul 24 19:41:25.275525 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.275493    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" event={"ID":"7c05249b-866f-4fd6-aa58-82e9c0f08d59","Type":"ContainerStarted","Data":"c08c08d73f1213cee4efbcba1cb763bd928bedc370a6618a014da76faeb239f8"}
Jul 24 19:41:25.441613 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.441585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttqdg\" (UniqueName: \"kubernetes.io/projected/811a2504-569a-4559-923a-d0a496f6e501-kube-api-access-ttqdg\") pod \"kuadrant-operator-catalog-jp5r8\" (UID: \"811a2504-569a-4559-923a-d0a496f6e501\") " pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:25.542111 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.541993    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttqdg\" (UniqueName: \"kubernetes.io/projected/811a2504-569a-4559-923a-d0a496f6e501-kube-api-access-ttqdg\") pod \"kuadrant-operator-catalog-jp5r8\" (UID: \"811a2504-569a-4559-923a-d0a496f6e501\") " pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:25.550640 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.550613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttqdg\" (UniqueName: \"kubernetes.io/projected/811a2504-569a-4559-923a-d0a496f6e501-kube-api-access-ttqdg\") pod \"kuadrant-operator-catalog-jp5r8\" (UID: \"811a2504-569a-4559-923a-d0a496f6e501\") " pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:25.564664 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.564635    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:25.744729 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:25.744696    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jp5r8"]
Jul 24 19:41:25.792414 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:41:25.792343    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811a2504_569a_4559_923a_d0a496f6e501.slice/crio-d4610a4503b259beb794b060a1f706999922857670e822c734d1716e56ee8146 WatchSource:0}: Error finding container d4610a4503b259beb794b060a1f706999922857670e822c734d1716e56ee8146: Status 404 returned error can't find the container with id d4610a4503b259beb794b060a1f706999922857670e822c734d1716e56ee8146
Jul 24 19:41:26.284187 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:26.284145    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8" event={"ID":"811a2504-569a-4559-923a-d0a496f6e501","Type":"ContainerStarted","Data":"d4610a4503b259beb794b060a1f706999922857670e822c734d1716e56ee8146"}
Jul 24 19:41:28.294989 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.294951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" event={"ID":"7c05249b-866f-4fd6-aa58-82e9c0f08d59","Type":"ContainerStarted","Data":"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"}
Jul 24 19:41:28.295454 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.294989    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" podUID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" containerName="registry-server" containerID="cri-o://0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e" gracePeriod=2
Jul 24 19:41:28.296521 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.296492    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8" event={"ID":"811a2504-569a-4559-923a-d0a496f6e501","Type":"ContainerStarted","Data":"91012e9ddae2cf2513d0d7919d99f61b26edae40d2abadd1193234dd4f450710"}
Jul 24 19:41:28.312344 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.312303    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" podStartSLOduration=2.216313822 podStartE2EDuration="4.312288802s" totalImagesPullingTime="2.09597498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:41:28.310846799 +0000 UTC m=+419.405392412" watchObservedRunningTime="2026-07-24 19:41:28.312288802 +0000 UTC m=+419.406834415"
Jul 24 19:41:28.537240 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.537214    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:28.554098 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.554006    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8" podStartSLOduration=2.121075218 podStartE2EDuration="3.55398816s" totalImagesPullingTime="1.432912942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:41:28.325387548 +0000 UTC m=+419.419933162" watchObservedRunningTime="2026-07-24 19:41:28.55398816 +0000 UTC m=+419.648533773"
Jul 24 19:41:28.667882 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.667845    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq5fk\" (UniqueName: \"kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk\") pod \"7c05249b-866f-4fd6-aa58-82e9c0f08d59\" (UID: \"7c05249b-866f-4fd6-aa58-82e9c0f08d59\") "
Jul 24 19:41:28.669954 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.669929    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk" (OuterVolumeSpecName: "kube-api-access-jq5fk") pod "7c05249b-866f-4fd6-aa58-82e9c0f08d59" (UID: "7c05249b-866f-4fd6-aa58-82e9c0f08d59"). InnerVolumeSpecName "kube-api-access-jq5fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:41:28.768967 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:28.768932    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq5fk\" (UniqueName: \"kubernetes.io/projected/7c05249b-866f-4fd6-aa58-82e9c0f08d59-kube-api-access-jq5fk\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:41:29.301409 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.301376    2568 generic.go:356] "Generic (PLEG): container finished" podID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" containerID="0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e" exitCode=0
Jul 24 19:41:29.301726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.301422    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jb46b"
Jul 24 19:41:29.301726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.301462    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" event={"ID":"7c05249b-866f-4fd6-aa58-82e9c0f08d59","Type":"ContainerDied","Data":"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"}
Jul 24 19:41:29.301726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.301517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jb46b" event={"ID":"7c05249b-866f-4fd6-aa58-82e9c0f08d59","Type":"ContainerDied","Data":"c08c08d73f1213cee4efbcba1cb763bd928bedc370a6618a014da76faeb239f8"}
Jul 24 19:41:29.301726 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.301545    2568 scope.go:117] "RemoveContainer" containerID="0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"
Jul 24 19:41:29.309915 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.309897    2568 scope.go:117] "RemoveContainer" containerID="0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"
Jul 24 19:41:29.310177 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:41:29.310158    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e\": container with ID starting with 0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e not found: ID does not exist" containerID="0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"
Jul 24 19:41:29.310234 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.310184    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e"} err="failed to get container status \"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e\": rpc error: code = NotFound desc = could not find container \"0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e\": container with ID starting with 0596f34182642b5a226c864ad81f0998ec1fcd6dc77809c1ba56835258920a8e not found: ID does not exist"
Jul 24 19:41:29.321593 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.321565    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:29.327270 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.327241    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jb46b"]
Jul 24 19:41:29.434695 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:29.434663    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" path="/var/lib/kubelet/pods/7c05249b-866f-4fd6-aa58-82e9c0f08d59/volumes"
Jul 24 19:41:35.565281 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:35.565222    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:35.565673 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:35.565319    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:35.586289 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:35.586248    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:36.350792 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:36.350764    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jp5r8"
Jul 24 19:41:55.977576 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.977498    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"]
Jul 24 19:41:55.983515 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.983374    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" containerName="registry-server"
Jul 24 19:41:55.983515 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.983511    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" containerName="registry-server"
Jul 24 19:41:55.983696 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.983582    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c05249b-866f-4fd6-aa58-82e9c0f08d59" containerName="registry-server"
Jul 24 19:41:55.989912 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.989882    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:55.990805 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.990781    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"]
Jul 24 19:41:55.992822 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.992803    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-s6rhc\""
Jul 24 19:41:55.992926 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:55.992862    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 19:41:56.090545 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:56.090514    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfqg\" (UniqueName: \"kubernetes.io/projected/2e282371-e8d2-46a5-8a23-3bab91d49af4-kube-api-access-fbfqg\") pod \"dns-operator-controller-manager-864dd78c4c-k9wqc\" (UID: \"2e282371-e8d2-46a5-8a23-3bab91d49af4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:56.191538 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:56.191493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfqg\" (UniqueName: \"kubernetes.io/projected/2e282371-e8d2-46a5-8a23-3bab91d49af4-kube-api-access-fbfqg\") pod \"dns-operator-controller-manager-864dd78c4c-k9wqc\" (UID: \"2e282371-e8d2-46a5-8a23-3bab91d49af4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:56.201759 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:56.201730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfqg\" (UniqueName: \"kubernetes.io/projected/2e282371-e8d2-46a5-8a23-3bab91d49af4-kube-api-access-fbfqg\") pod \"dns-operator-controller-manager-864dd78c4c-k9wqc\" (UID: \"2e282371-e8d2-46a5-8a23-3bab91d49af4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:56.301022 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:56.300941    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:56.430944 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:56.430920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"]
Jul 24 19:41:56.433313 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:41:56.433283    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e282371_e8d2_46a5_8a23_3bab91d49af4.slice/crio-60dfe2a958f2e2180f45989168970dd5596a7269014f85469ebc236e69862d83 WatchSource:0}: Error finding container 60dfe2a958f2e2180f45989168970dd5596a7269014f85469ebc236e69862d83: Status 404 returned error can't find the container with id 60dfe2a958f2e2180f45989168970dd5596a7269014f85469ebc236e69862d83
Jul 24 19:41:57.431303 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:57.431249    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc" event={"ID":"2e282371-e8d2-46a5-8a23-3bab91d49af4","Type":"ContainerStarted","Data":"60dfe2a958f2e2180f45989168970dd5596a7269014f85469ebc236e69862d83"}
Jul 24 19:41:58.438171 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:58.438139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc" event={"ID":"2e282371-e8d2-46a5-8a23-3bab91d49af4","Type":"ContainerStarted","Data":"82ae11c4ba8b52a2881c9f859ab27ec97ccec0b4d9409a7c907e4083cfe5227f"}
Jul 24 19:41:58.438633 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:58.438201    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:41:58.454293 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:41:58.454228    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc" podStartSLOduration=1.565429843 podStartE2EDuration="3.454213017s" totalImagesPullingTime="1.888783174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:41:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:41:58.453774336 +0000 UTC m=+449.548320032" watchObservedRunningTime="2026-07-24 19:41:58.454213017 +0000 UTC m=+449.548758630"
Jul 24 19:42:01.327956 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.327924    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"]
Jul 24 19:42:01.334222 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.334196    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.337129 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.337113    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dztm7\""
Jul 24 19:42:01.344564 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.344539    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"]
Jul 24 19:42:01.429719 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.429682    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.429719 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.429728    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6svs\" (UniqueName: \"kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.530310 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.530248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.530494 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.530336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6svs\" (UniqueName: \"kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.530753 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.530724    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.544268 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.544230    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6svs\" (UniqueName: \"kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs\") pod \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.644349 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.644269    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:01.807391 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:01.807354    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"]
Jul 24 19:42:01.810868 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:01.810842    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9480b41_4b33_42c3_b492_1dd6c5e0b531.slice/crio-6bf10be879590c1b3f229af358ff9ddd684cf6b364e6a4bb4562b358d282b67f WatchSource:0}: Error finding container 6bf10be879590c1b3f229af358ff9ddd684cf6b364e6a4bb4562b358d282b67f: Status 404 returned error can't find the container with id 6bf10be879590c1b3f229af358ff9ddd684cf6b364e6a4bb4562b358d282b67f
Jul 24 19:42:02.464704 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:02.464671    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm" event={"ID":"e9480b41-4b33-42c3-b492-1dd6c5e0b531","Type":"ContainerStarted","Data":"6bf10be879590c1b3f229af358ff9ddd684cf6b364e6a4bb4562b358d282b67f"}
Jul 24 19:42:04.787087 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.787033    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"]
Jul 24 19:42:04.790602 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.790584    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:04.793611 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.793584    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-snxct\""
Jul 24 19:42:04.800551 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.800528    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"]
Jul 24 19:42:04.863352 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.863309    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thm6x\" (UniqueName: \"kubernetes.io/projected/15c8e78c-5673-45a7-880c-3d0884d2610d-kube-api-access-thm6x\") pod \"authorino-operator-75cc8d5cb7-cwzs6\" (UID: \"15c8e78c-5673-45a7-880c-3d0884d2610d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:04.964667 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.964624    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thm6x\" (UniqueName: \"kubernetes.io/projected/15c8e78c-5673-45a7-880c-3d0884d2610d-kube-api-access-thm6x\") pod \"authorino-operator-75cc8d5cb7-cwzs6\" (UID: \"15c8e78c-5673-45a7-880c-3d0884d2610d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:04.986692 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:04.986658    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thm6x\" (UniqueName: \"kubernetes.io/projected/15c8e78c-5673-45a7-880c-3d0884d2610d-kube-api-access-thm6x\") pod \"authorino-operator-75cc8d5cb7-cwzs6\" (UID: \"15c8e78c-5673-45a7-880c-3d0884d2610d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:05.102282 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:05.102182    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:05.255371 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:05.255344    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"]
Jul 24 19:42:05.257403 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:05.257365    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c8e78c_5673_45a7_880c_3d0884d2610d.slice/crio-a282b4785cf33a8640aa061b07d333bfe86c4c85736ea30d8f8deb9182f140bf WatchSource:0}: Error finding container a282b4785cf33a8640aa061b07d333bfe86c4c85736ea30d8f8deb9182f140bf: Status 404 returned error can't find the container with id a282b4785cf33a8640aa061b07d333bfe86c4c85736ea30d8f8deb9182f140bf
Jul 24 19:42:05.479665 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:05.479621    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6" event={"ID":"15c8e78c-5673-45a7-880c-3d0884d2610d","Type":"ContainerStarted","Data":"a282b4785cf33a8640aa061b07d333bfe86c4c85736ea30d8f8deb9182f140bf"}
Jul 24 19:42:07.519741 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.519708    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"]
Jul 24 19:42:07.528388 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.528364    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:07.532014 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.531990    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ck6t7\""
Jul 24 19:42:07.539125 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.539098    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"]
Jul 24 19:42:07.689054 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.689011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnxrp\" (UniqueName: \"kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp\") pod \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" (UID: \"1a0ae9f4-b30f-4f28-9087-46ae61897138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:07.790062 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.789944    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnxrp\" (UniqueName: \"kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp\") pod \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" (UID: \"1a0ae9f4-b30f-4f28-9087-46ae61897138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:07.799113 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.799078    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnxrp\" (UniqueName: \"kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp\") pod \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" (UID: \"1a0ae9f4-b30f-4f28-9087-46ae61897138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:07.841435 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:07.841399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:08.275232 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.275206    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"]
Jul 24 19:42:08.278846 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:08.278817    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0ae9f4_b30f_4f28_9087_46ae61897138.slice/crio-4861d3679c655dde4db0860b5124bd8bfd6933ac65163f9a4ad92324698c80fd WatchSource:0}: Error finding container 4861d3679c655dde4db0860b5124bd8bfd6933ac65163f9a4ad92324698c80fd: Status 404 returned error can't find the container with id 4861d3679c655dde4db0860b5124bd8bfd6933ac65163f9a4ad92324698c80fd
Jul 24 19:42:08.496482 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.496447    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s" event={"ID":"1a0ae9f4-b30f-4f28-9087-46ae61897138","Type":"ContainerStarted","Data":"4861d3679c655dde4db0860b5124bd8bfd6933ac65163f9a4ad92324698c80fd"}
Jul 24 19:42:08.497988 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.497953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm" event={"ID":"e9480b41-4b33-42c3-b492-1dd6c5e0b531","Type":"ContainerStarted","Data":"9cf601d576eba42185e298b7b10c88686e1dd001db66d83eebd4f162451b8b74"}
Jul 24 19:42:08.498115 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.498028    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:08.499407 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.499381    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6" event={"ID":"15c8e78c-5673-45a7-880c-3d0884d2610d","Type":"ContainerStarted","Data":"5d1fb080e0d688dc7745078a7f42740683f45993d0bdaec361588663e3c98afa"}
Jul 24 19:42:08.499515 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.499501    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:08.518917 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.518860    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm" podStartSLOduration=1.176055291 podStartE2EDuration="7.518841774s" totalImagesPullingTime="6.342786483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:08.517113143 +0000 UTC m=+459.611658757" watchObservedRunningTime="2026-07-24 19:42:08.518841774 +0000 UTC m=+459.613387389"
Jul 24 19:42:08.532446 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:08.532400    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6" podStartSLOduration=1.597585897 podStartE2EDuration="4.532386408s" totalImagesPullingTime="2.934800511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:08.531905277 +0000 UTC m=+459.626450884" watchObservedRunningTime="2026-07-24 19:42:08.532386408 +0000 UTC m=+459.626932020"
Jul 24 19:42:09.445320 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:09.445252    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-k9wqc"
Jul 24 19:42:10.510311 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:10.510252    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s" event={"ID":"1a0ae9f4-b30f-4f28-9087-46ae61897138","Type":"ContainerStarted","Data":"e8dba34ba455d000370752c51ebc5dd8367d8016abd0d09310b6225c964e8c61"}
Jul 24 19:42:10.510684 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:10.510429    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:10.531487 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:10.531435    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s" podStartSLOduration=1.895529695 podStartE2EDuration="3.531422819s" totalImagesPullingTime="1.635893124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:10.529841425 +0000 UTC m=+461.624387039" watchObservedRunningTime="2026-07-24 19:42:10.531422819 +0000 UTC m=+461.625968431"
Jul 24 19:42:19.506715 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:19.506680    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwzs6"
Jul 24 19:42:19.507164 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:19.506785    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.336626 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.336592    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"]
Jul 24 19:42:21.337014 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.336809    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" containerName="manager" containerID="cri-o://9cf601d576eba42185e298b7b10c88686e1dd001db66d83eebd4f162451b8b74" gracePeriod=2
Jul 24 19:42:21.339713 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.339676    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.340896 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.340865    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"]
Jul 24 19:42:21.355763 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.355686    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"]
Jul 24 19:42:21.355952 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.355920    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" containerName="manager" containerID="cri-o://e8dba34ba455d000370752c51ebc5dd8367d8016abd0d09310b6225c964e8c61" gracePeriod=2
Jul 24 19:42:21.358088 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.358028    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:21.358190 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.358172    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:21.358797 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.358774    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" containerName="manager"
Jul 24 19:42:21.358797 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.358798    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" containerName="manager"
Jul 24 19:42:21.358960 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.358880    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" containerName="manager"
Jul 24 19:42:21.361032 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.360996    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.362279 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.362243    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.362424 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.362283    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"]
Jul 24 19:42:21.363941 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.363908    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:21.378060 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.378031    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:21.383246 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.383224    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"]
Jul 24 19:42:21.383566 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.383552    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" containerName="manager"
Jul 24 19:42:21.383608 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.383567    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" containerName="manager"
Jul 24 19:42:21.383654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.383630    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" containerName="manager"
Jul 24 19:42:21.386493 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.386478    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:21.400727 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.400657    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"]
Jul 24 19:42:21.406168 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.406090    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:21.409206 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.409181    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.431418 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.431377    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:21.434060 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:21.434026    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-s7ptm\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.459225 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.459197    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:42:21.463024 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.463001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.479199 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.478957    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:42:21.512666 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.512616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98n4n\" (UniqueName: \"kubernetes.io/projected/b4f2f490-2b44-42a3-a21a-173602782e30-kube-api-access-98n4n\") pod \"limitador-operator-controller-manager-859d7b78bd-h9njb\" (UID: \"b4f2f490-2b44-42a3-a21a-173602782e30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:21.512817 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.512762    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.512817 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.512792    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95d49\" (UniqueName: \"kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.561732 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.561698    2568 generic.go:356] "Generic (PLEG): container finished" podID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" containerID="9cf601d576eba42185e298b7b10c88686e1dd001db66d83eebd4f162451b8b74" exitCode=0
Jul 24 19:42:21.563555 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.563522    2568 generic.go:356] "Generic (PLEG): container finished" podID="1a0ae9f4-b30f-4f28-9087-46ae61897138" containerID="e8dba34ba455d000370752c51ebc5dd8367d8016abd0d09310b6225c964e8c61" exitCode=0
Jul 24 19:42:21.602396 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.602373    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:21.604756 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.604737    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:21.613292 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.613350 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613318    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.613393 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613350    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95d49\" (UniqueName: \"kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.613393 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613379    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98n4n\" (UniqueName: \"kubernetes.io/projected/b4f2f490-2b44-42a3-a21a-173602782e30-kube-api-access-98n4n\") pod \"limitador-operator-controller-manager-859d7b78bd-h9njb\" (UID: \"b4f2f490-2b44-42a3-a21a-173602782e30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:21.613492 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613403    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-529np\" (UniqueName: \"kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.613652 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.613634    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.623404 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.623374    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95d49\" (UniqueName: \"kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5fh2n\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.623572 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.623551    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98n4n\" (UniqueName: \"kubernetes.io/projected/b4f2f490-2b44-42a3-a21a-173602782e30-kube-api-access-98n4n\") pod \"limitador-operator-controller-manager-859d7b78bd-h9njb\" (UID: \"b4f2f490-2b44-42a3-a21a-173602782e30\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:21.714628 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.714593    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume\") pod \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") "
Jul 24 19:42:21.714812 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.714668    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnxrp\" (UniqueName: \"kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp\") pod \"1a0ae9f4-b30f-4f28-9087-46ae61897138\" (UID: \"1a0ae9f4-b30f-4f28-9087-46ae61897138\") "
Jul 24 19:42:21.714812 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.714753    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6svs\" (UniqueName: \"kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs\") pod \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\" (UID: \"e9480b41-4b33-42c3-b492-1dd6c5e0b531\") "
Jul 24 19:42:21.714931 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.714914    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.715001 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.714985    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-529np\" (UniqueName: \"kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.715119 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.715084    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e9480b41-4b33-42c3-b492-1dd6c5e0b531" (UID: "e9480b41-4b33-42c3-b492-1dd6c5e0b531"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:42:21.715324 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.715303    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.716747 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.716726    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp" (OuterVolumeSpecName: "kube-api-access-lnxrp") pod "1a0ae9f4-b30f-4f28-9087-46ae61897138" (UID: "1a0ae9f4-b30f-4f28-9087-46ae61897138"). InnerVolumeSpecName "kube-api-access-lnxrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:42:21.716824 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.716806    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs" (OuterVolumeSpecName: "kube-api-access-n6svs") pod "e9480b41-4b33-42c3-b492-1dd6c5e0b531" (UID: "e9480b41-4b33-42c3-b492-1dd6c5e0b531"). InnerVolumeSpecName "kube-api-access-n6svs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:42:21.727150 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.727129    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-529np\" (UniqueName: \"kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np\") pod \"kuadrant-operator-controller-manager-86878c769f-wdxlc\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.767165 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.767125    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:21.775265 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.775236    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:21.783999 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.783977    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:21.816425 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.816386    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnxrp\" (UniqueName: \"kubernetes.io/projected/1a0ae9f4-b30f-4f28-9087-46ae61897138-kube-api-access-lnxrp\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:21.816425 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.816423    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6svs\" (UniqueName: \"kubernetes.io/projected/e9480b41-4b33-42c3-b492-1dd6c5e0b531-kube-api-access-n6svs\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:21.816654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.816439    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9480b41-4b33-42c3-b492-1dd6c5e0b531-extensions-socket-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:21.922143 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:21.922117    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:21.924478 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:21.924448    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6776dc_cc36_4749_bfff_e85f01f8d50b.slice/crio-a2ab522a20288187901971ccdf30d6125f39981283666bd59fee85b7bd802fb2 WatchSource:0}: Error finding container a2ab522a20288187901971ccdf30d6125f39981283666bd59fee85b7bd802fb2: Status 404 returned error can't find the container with id a2ab522a20288187901971ccdf30d6125f39981283666bd59fee85b7bd802fb2
Jul 24 19:42:22.152765 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.152738    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:42:22.155755 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:22.155724    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9fbbf7_7fd6_42b5_9078_2d555659b617.slice/crio-3c1fc6c0843ba49c085f46ba65842e9a960370632d6abb2c0242411afceea11c WatchSource:0}: Error finding container 3c1fc6c0843ba49c085f46ba65842e9a960370632d6abb2c0242411afceea11c: Status 404 returned error can't find the container with id 3c1fc6c0843ba49c085f46ba65842e9a960370632d6abb2c0242411afceea11c
Jul 24 19:42:22.156358 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.156328    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"]
Jul 24 19:42:22.158074 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:22.157992    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f2f490_2b44_42a3_a21a_173602782e30.slice/crio-f23bb771d08cd622becde72b022cb09e02375624790c4462ec814761e099a62e WatchSource:0}: Error finding container f23bb771d08cd622becde72b022cb09e02375624790c4462ec814761e099a62e: Status 404 returned error can't find the container with id f23bb771d08cd622becde72b022cb09e02375624790c4462ec814761e099a62e
Jul 24 19:42:22.569581 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.569538    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" event={"ID":"1a6776dc-cc36-4749-bfff-e85f01f8d50b","Type":"ContainerStarted","Data":"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"}
Jul 24 19:42:22.570026 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.569601    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" event={"ID":"1a6776dc-cc36-4749-bfff-e85f01f8d50b","Type":"ContainerStarted","Data":"a2ab522a20288187901971ccdf30d6125f39981283666bd59fee85b7bd802fb2"}
Jul 24 19:42:22.570026 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.569628    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:22.571237 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.571210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb" event={"ID":"b4f2f490-2b44-42a3-a21a-173602782e30","Type":"ContainerStarted","Data":"2aa6ce2779c9c7615f1362b255a2c17f724c453d079c3a262eff3a1091efb50e"}
Jul 24 19:42:22.571391 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.571251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb" event={"ID":"b4f2f490-2b44-42a3-a21a-173602782e30","Type":"ContainerStarted","Data":"f23bb771d08cd622becde72b022cb09e02375624790c4462ec814761e099a62e"}
Jul 24 19:42:22.571391 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.571325    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:22.572827 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.572807    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" event={"ID":"6a9fbbf7-7fd6-42b5-9078-2d555659b617","Type":"ContainerStarted","Data":"ab7b73528edfb665933125b3f499819275e54037dd73395e4c6bb04d3f93e13e"}
Jul 24 19:42:22.572918 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.572837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" event={"ID":"6a9fbbf7-7fd6-42b5-9078-2d555659b617","Type":"ContainerStarted","Data":"3c1fc6c0843ba49c085f46ba65842e9a960370632d6abb2c0242411afceea11c"}
Jul 24 19:42:22.572965 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.572931    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:22.574175 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.574160    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-s7ptm"
Jul 24 19:42:22.574253 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.574192    2568 scope.go:117] "RemoveContainer" containerID="9cf601d576eba42185e298b7b10c88686e1dd001db66d83eebd4f162451b8b74"
Jul 24 19:42:22.575579 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.575558    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:22.582796 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.582776    2568 scope.go:117] "RemoveContainer" containerID="e8dba34ba455d000370752c51ebc5dd8367d8016abd0d09310b6225c964e8c61"
Jul 24 19:42:22.590121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.590077    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" podStartSLOduration=1.590050084 podStartE2EDuration="1.590050084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:22.588901945 +0000 UTC m=+473.683447558" watchObservedRunningTime="2026-07-24 19:42:22.590050084 +0000 UTC m=+473.684595696"
Jul 24 19:42:22.608980 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.608935    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb" podStartSLOduration=1.60891831 podStartE2EDuration="1.60891831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:22.608571432 +0000 UTC m=+473.703117045" watchObservedRunningTime="2026-07-24 19:42:22.60891831 +0000 UTC m=+473.703463923"
Jul 24 19:42:22.630659 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:22.630614    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" podStartSLOduration=1.63059857 podStartE2EDuration="1.63059857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:22.629134737 +0000 UTC m=+473.723680352" watchObservedRunningTime="2026-07-24 19:42:22.63059857 +0000 UTC m=+473.725144183"
Jul 24 19:42:22.632073 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:22.632050    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tqz2s\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tqz2s"
Jul 24 19:42:23.429808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:23.429777    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a0ae9f4-b30f-4f28-9087-46ae61897138" path="/var/lib/kubelet/pods/1a0ae9f4-b30f-4f28-9087-46ae61897138/volumes"
Jul 24 19:42:23.430138 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:23.430096    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9480b41-4b33-42c3-b492-1dd6c5e0b531" path="/var/lib/kubelet/pods/e9480b41-4b33-42c3-b492-1dd6c5e0b531/volumes"
Jul 24 19:42:33.589829 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.589792    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h9njb"
Jul 24 19:42:33.595943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.589861    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:33.595943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.589891    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:42:33.676572 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.676539    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:33.676808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.676761    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" podUID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" containerName="manager" containerID="cri-o://fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90" gracePeriod=10
Jul 24 19:42:33.951851 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:33.951816    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:34.015601 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.015569    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95d49\" (UniqueName: \"kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49\") pod \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") "
Jul 24 19:42:34.015749 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.015623    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume\") pod \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\" (UID: \"1a6776dc-cc36-4749-bfff-e85f01f8d50b\") "
Jul 24 19:42:34.016079 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.016046    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1a6776dc-cc36-4749-bfff-e85f01f8d50b" (UID: "1a6776dc-cc36-4749-bfff-e85f01f8d50b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:42:34.018037 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.018009    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49" (OuterVolumeSpecName: "kube-api-access-95d49") pod "1a6776dc-cc36-4749-bfff-e85f01f8d50b" (UID: "1a6776dc-cc36-4749-bfff-e85f01f8d50b"). InnerVolumeSpecName "kube-api-access-95d49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:42:34.117252 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.117170    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95d49\" (UniqueName: \"kubernetes.io/projected/1a6776dc-cc36-4749-bfff-e85f01f8d50b-kube-api-access-95d49\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:34.117252 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.117203    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6776dc-cc36-4749-bfff-e85f01f8d50b-extensions-socket-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:34.638478 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.638440    2568 generic.go:356] "Generic (PLEG): container finished" podID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" containerID="fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90" exitCode=0
Jul 24 19:42:34.638883 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.638502    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"
Jul 24 19:42:34.638883 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.638508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" event={"ID":"1a6776dc-cc36-4749-bfff-e85f01f8d50b","Type":"ContainerDied","Data":"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"}
Jul 24 19:42:34.638883 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.638564    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n" event={"ID":"1a6776dc-cc36-4749-bfff-e85f01f8d50b","Type":"ContainerDied","Data":"a2ab522a20288187901971ccdf30d6125f39981283666bd59fee85b7bd802fb2"}
Jul 24 19:42:34.638883 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.638593    2568 scope.go:117] "RemoveContainer" containerID="fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"
Jul 24 19:42:34.647403 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.647383    2568 scope.go:117] "RemoveContainer" containerID="fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"
Jul 24 19:42:34.647676 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:34.647655    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90\": container with ID starting with fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90 not found: ID does not exist" containerID="fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"
Jul 24 19:42:34.647722 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.647685    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90"} err="failed to get container status \"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90\": rpc error: code = NotFound desc = could not find container \"fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90\": container with ID starting with fdd8591fda5b0022a274c697a88fe7c66779402aa8181786c2ea64a2d408fa90 not found: ID does not exist"
Jul 24 19:42:34.661124 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.661095    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:34.669335 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:34.669311    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5fh2n"]
Jul 24 19:42:35.430785 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:35.430751    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" path="/var/lib/kubelet/pods/1a6776dc-cc36-4749-bfff-e85f01f8d50b/volumes"
Jul 24 19:42:51.873629 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.873595    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:42:51.874067 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.873957    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" containerName="manager"
Jul 24 19:42:51.874067 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.873971    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" containerName="manager"
Jul 24 19:42:51.874067 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.874023    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a6776dc-cc36-4749-bfff-e85f01f8d50b" containerName="manager"
Jul 24 19:42:51.876717 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.876695    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:51.879372 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.879350    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 19:42:51.879521 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.879501    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tf6ws\""
Jul 24 19:42:51.887223 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.887198    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:42:51.977660 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.977625    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:42:51.977950 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.977923    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6nmh\" (UniqueName: \"kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:51.978029 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:51.977986    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.078580 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.078545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.078764 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.078609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6nmh\" (UniqueName: \"kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.079322 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.079298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.086820 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.086792    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6nmh\" (UniqueName: \"kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh\") pod \"limitador-limitador-64c94fd64f-f8jcm\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.188135 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.188040    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:52.328505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.328480    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:42:52.330061 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:52.330030    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69df33d6_fb6f_49f2_9860_349a0ac7a4f8.slice/crio-16100ed8d585356c8a03ec4a911ee8384448dba57d0a3fb1203309ce8c163f03 WatchSource:0}: Error finding container 16100ed8d585356c8a03ec4a911ee8384448dba57d0a3fb1203309ce8c163f03: Status 404 returned error can't find the container with id 16100ed8d585356c8a03ec4a911ee8384448dba57d0a3fb1203309ce8c163f03
Jul 24 19:42:52.664756 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.664716    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:42:52.666699 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.666682    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:52.673989 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.673957    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6c2s4\""
Jul 24 19:42:52.675840 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.675808    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:42:52.719642 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.719600    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" event={"ID":"69df33d6-fb6f-49f2-9860-349a0ac7a4f8","Type":"ContainerStarted","Data":"16100ed8d585356c8a03ec4a911ee8384448dba57d0a3fb1203309ce8c163f03"}
Jul 24 19:42:52.784401 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.784361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcqkd\" (UniqueName: \"kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd\") pod \"authorino-58cd955675-kc7vc\" (UID: \"6a2d5b63-8710-43ab-a8b3-e681ebf72540\") " pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:52.885001 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.884960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcqkd\" (UniqueName: \"kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd\") pod \"authorino-58cd955675-kc7vc\" (UID: \"6a2d5b63-8710-43ab-a8b3-e681ebf72540\") " pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:52.893872 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.893842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcqkd\" (UniqueName: \"kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd\") pod \"authorino-58cd955675-kc7vc\" (UID: \"6a2d5b63-8710-43ab-a8b3-e681ebf72540\") " pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:52.983662 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:52.983629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:53.144512 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:53.144474    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:42:53.148930 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:42:53.148886    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a2d5b63_8710_43ab_a8b3_e681ebf72540.slice/crio-806086583d23bb6585a1f25b95e93730fd599ffe7669e355203b8cb306cd4d59 WatchSource:0}: Error finding container 806086583d23bb6585a1f25b95e93730fd599ffe7669e355203b8cb306cd4d59: Status 404 returned error can't find the container with id 806086583d23bb6585a1f25b95e93730fd599ffe7669e355203b8cb306cd4d59
Jul 24 19:42:53.726244 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:53.726206    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kc7vc" event={"ID":"6a2d5b63-8710-43ab-a8b3-e681ebf72540","Type":"ContainerStarted","Data":"806086583d23bb6585a1f25b95e93730fd599ffe7669e355203b8cb306cd4d59"}
Jul 24 19:42:57.258135 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.258101    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:42:57.747877 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.747824    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kc7vc" event={"ID":"6a2d5b63-8710-43ab-a8b3-e681ebf72540","Type":"ContainerStarted","Data":"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"}
Jul 24 19:42:57.749542 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.749513    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" event={"ID":"69df33d6-fb6f-49f2-9860-349a0ac7a4f8","Type":"ContainerStarted","Data":"d85451803ccf2e931893e78efd0f55fe60747a4f9e2a360e215e8e724b028596"}
Jul 24 19:42:57.749672 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.749653    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:42:57.763825 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.763775    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kc7vc" podStartSLOduration=2.103692329 podStartE2EDuration="5.76375985s" totalImagesPullingTime="3.660067521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:57.762694087 +0000 UTC m=+508.857239699" watchObservedRunningTime="2026-07-24 19:42:57.76375985 +0000 UTC m=+508.858305454"
Jul 24 19:42:57.780236 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:57.780180    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" podStartSLOduration=2.339865629 podStartE2EDuration="6.780163832s" totalImagesPullingTime="4.440298203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:42:57.778767324 +0000 UTC m=+508.873312938" watchObservedRunningTime="2026-07-24 19:42:57.780163832 +0000 UTC m=+508.874709461"
Jul 24 19:42:58.754121 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:58.754081    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kc7vc" podUID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" containerName="authorino" containerID="cri-o://8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3" gracePeriod=30
Jul 24 19:42:58.995430 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:58.995402    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:59.143066 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.142978    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcqkd\" (UniqueName: \"kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd\") pod \"6a2d5b63-8710-43ab-a8b3-e681ebf72540\" (UID: \"6a2d5b63-8710-43ab-a8b3-e681ebf72540\") "
Jul 24 19:42:59.145001 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.144970    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd" (OuterVolumeSpecName: "kube-api-access-dcqkd") pod "6a2d5b63-8710-43ab-a8b3-e681ebf72540" (UID: "6a2d5b63-8710-43ab-a8b3-e681ebf72540"). InnerVolumeSpecName "kube-api-access-dcqkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:42:59.244532 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.244491    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcqkd\" (UniqueName: \"kubernetes.io/projected/6a2d5b63-8710-43ab-a8b3-e681ebf72540-kube-api-access-dcqkd\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:42:59.759845 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.759811    2568 generic.go:356] "Generic (PLEG): container finished" podID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" containerID="8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3" exitCode=0
Jul 24 19:42:59.760302 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.759856    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kc7vc"
Jul 24 19:42:59.760302 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.759875    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kc7vc" event={"ID":"6a2d5b63-8710-43ab-a8b3-e681ebf72540","Type":"ContainerDied","Data":"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"}
Jul 24 19:42:59.760302 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.759908    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kc7vc" event={"ID":"6a2d5b63-8710-43ab-a8b3-e681ebf72540","Type":"ContainerDied","Data":"806086583d23bb6585a1f25b95e93730fd599ffe7669e355203b8cb306cd4d59"}
Jul 24 19:42:59.760302 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.759925    2568 scope.go:117] "RemoveContainer" containerID="8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"
Jul 24 19:42:59.768496 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.768472    2568 scope.go:117] "RemoveContainer" containerID="8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"
Jul 24 19:42:59.768754 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:42:59.768734    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3\": container with ID starting with 8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3 not found: ID does not exist" containerID="8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"
Jul 24 19:42:59.768813 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.768764    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3"} err="failed to get container status \"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3\": rpc error: code = NotFound desc = could not find container \"8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3\": container with ID starting with 8ba01c1276478a991ac4e0d311c386ceec2193d04a8e8f21d96fe389e9654ae3 not found: ID does not exist"
Jul 24 19:42:59.778107 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.778049    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:42:59.779909 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:42:59.779885    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kc7vc"]
Jul 24 19:43:01.430846 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:01.430812    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" path="/var/lib/kubelet/pods/6a2d5b63-8710-43ab-a8b3-e681ebf72540/volumes"
Jul 24 19:43:08.269197 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.269167    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:43:08.269609 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.269382    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" podUID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" containerName="limitador" containerID="cri-o://d85451803ccf2e931893e78efd0f55fe60747a4f9e2a360e215e8e724b028596" gracePeriod=30
Jul 24 19:43:08.270128 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.270110    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:43:08.802649 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.802618    2568 generic.go:356] "Generic (PLEG): container finished" podID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" containerID="d85451803ccf2e931893e78efd0f55fe60747a4f9e2a360e215e8e724b028596" exitCode=0
Jul 24 19:43:08.802808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.802704    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" event={"ID":"69df33d6-fb6f-49f2-9860-349a0ac7a4f8","Type":"ContainerDied","Data":"d85451803ccf2e931893e78efd0f55fe60747a4f9e2a360e215e8e724b028596"}
Jul 24 19:43:08.824993 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.824971    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:43:08.924018 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.923930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6nmh\" (UniqueName: \"kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh\") pod \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") "
Jul 24 19:43:08.924018 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.923972    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file\") pod \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\" (UID: \"69df33d6-fb6f-49f2-9860-349a0ac7a4f8\") "
Jul 24 19:43:08.924431 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.924402    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file" (OuterVolumeSpecName: "config-file") pod "69df33d6-fb6f-49f2-9860-349a0ac7a4f8" (UID: "69df33d6-fb6f-49f2-9860-349a0ac7a4f8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 19:43:08.925998 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:08.925977    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh" (OuterVolumeSpecName: "kube-api-access-v6nmh") pod "69df33d6-fb6f-49f2-9860-349a0ac7a4f8" (UID: "69df33d6-fb6f-49f2-9860-349a0ac7a4f8"). InnerVolumeSpecName "kube-api-access-v6nmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:43:09.025211 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.025176    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6nmh\" (UniqueName: \"kubernetes.io/projected/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-kube-api-access-v6nmh\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:09.025211 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.025206    2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/69df33d6-fb6f-49f2-9860-349a0ac7a4f8-config-file\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:09.808689 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.808595    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm" event={"ID":"69df33d6-fb6f-49f2-9860-349a0ac7a4f8","Type":"ContainerDied","Data":"16100ed8d585356c8a03ec4a911ee8384448dba57d0a3fb1203309ce8c163f03"}
Jul 24 19:43:09.808689 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.808646    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"
Jul 24 19:43:09.808689 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.808676    2568 scope.go:117] "RemoveContainer" containerID="d85451803ccf2e931893e78efd0f55fe60747a4f9e2a360e215e8e724b028596"
Jul 24 19:43:09.824647 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.824617    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:43:09.826246 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:09.826213    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f8jcm"]
Jul 24 19:43:11.430984 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:11.430946    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" path="/var/lib/kubelet/pods/69df33d6-fb6f-49f2-9860-349a0ac7a4f8/volumes"
Jul 24 19:43:13.546146 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546115    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"]
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546472    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" containerName="limitador"
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546483    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" containerName="limitador"
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546500    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" containerName="authorino"
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546506    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" containerName="authorino"
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546560    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="69df33d6-fb6f-49f2-9860-349a0ac7a4f8" containerName="limitador"
Jul 24 19:43:13.546614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.546568    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2d5b63-8710-43ab-a8b3-e681ebf72540" containerName="authorino"
Jul 24 19:43:13.549189 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.549170    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.552005 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.551983    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 19:43:13.552116 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.551988    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 19:43:13.553288 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.553249    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 19:43:13.553374 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.553297    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xhw66\""
Jul 24 19:43:13.558028 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.558005    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"]
Jul 24 19:43:13.666450 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.666414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-data\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.666644 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.666522    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwkr\" (UniqueName: \"kubernetes.io/projected/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-kube-api-access-4qwkr\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.767200 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.767160    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwkr\" (UniqueName: \"kubernetes.io/projected/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-kube-api-access-4qwkr\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.767389 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.767233    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-data\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.767624 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.767603    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-data\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.775718 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.775692    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwkr\" (UniqueName: \"kubernetes.io/projected/d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b-kube-api-access-4qwkr\") pod \"postgres-69fb4594cf-wnvrp\" (UID: \"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:13.860520 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:13.860433    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:14.005189 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:14.005140    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"]
Jul 24 19:43:14.009188 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:43:14.009151    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98cdbf6_fe6a_43bc_87f9_ceeed9e0649b.slice/crio-a833906905e19bd5bed743c276c355afd74847eb25dafe0ca5ca9df0f412bc1f WatchSource:0}: Error finding container a833906905e19bd5bed743c276c355afd74847eb25dafe0ca5ca9df0f412bc1f: Status 404 returned error can't find the container with id a833906905e19bd5bed743c276c355afd74847eb25dafe0ca5ca9df0f412bc1f
Jul 24 19:43:14.840027 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:14.839954    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp" event={"ID":"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b","Type":"ContainerStarted","Data":"a833906905e19bd5bed743c276c355afd74847eb25dafe0ca5ca9df0f412bc1f"}
Jul 24 19:43:18.861435 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:18.861399    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp" event={"ID":"d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b","Type":"ContainerStarted","Data":"d9c58b873d07cb95a9e736a2f6f22c279d4583ce2e22b823dc67a265eee18114"}
Jul 24 19:43:18.861853 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:18.861482    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:18.879985 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:18.879920    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp" podStartSLOduration=1.131660453 podStartE2EDuration="5.879898537s" totalImagesPullingTime="4.748238084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:43:18.877701846 +0000 UTC m=+529.972247459" watchObservedRunningTime="2026-07-24 19:43:18.879898537 +0000 UTC m=+529.974444153"
Jul 24 19:43:24.895398 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:24.895310    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnvrp"
Jul 24 19:43:25.441598 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.441565    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:25.447682 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.447657    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:25.450514 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.450486    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6c2s4\""
Jul 24 19:43:25.452130 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.452101    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:25.458091 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.458066    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn92k\" (UniqueName: \"kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k\") pod \"authorino-5d4b67c446-djbth\" (UID: \"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35\") " pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:25.559350 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.559308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn92k\" (UniqueName: \"kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k\") pod \"authorino-5d4b67c446-djbth\" (UID: \"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35\") " pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:25.567855 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.567819    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn92k\" (UniqueName: \"kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k\") pod \"authorino-5d4b67c446-djbth\" (UID: \"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35\") " pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:25.618966 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.618928    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:25.619193 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.619180    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:25.645097 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.645065    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-s7czc"]
Jul 24 19:43:25.652174 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.652145    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.656140 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.655949    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 19:43:25.656784 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.656724    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-s7czc"]
Jul 24 19:43:25.677577 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.677541    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s7czc"]
Jul 24 19:43:25.677863 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:43:25.677835    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7jswx tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-7jswx tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-s7czc" podUID="f30c952f-6e80-4c93-8275-f87751f145d1"
Jul 24 19:43:25.705068 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.705016    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:43:25.707526 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.707499    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.716292 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.716220    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:43:25.761152 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.761119    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5sp7\" (UniqueName: \"kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.761329 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.761165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jswx\" (UniqueName: \"kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.761329 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.761244    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.761329 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.761306    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.769159 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.769136    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:25.771659 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:43:25.771628    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe7303c9_bab1_49a8_bb03_5aaa0d3caa35.slice/crio-58ca385150447678d8af4abf0112ae6d336c0cf381033517b53e94596d56437a WatchSource:0}: Error finding container 58ca385150447678d8af4abf0112ae6d336c0cf381033517b53e94596d56437a: Status 404 returned error can't find the container with id 58ca385150447678d8af4abf0112ae6d336c0cf381033517b53e94596d56437a
Jul 24 19:43:25.862409 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.862371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.862409 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.862411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.862654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.862459    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5sp7\" (UniqueName: \"kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.862654 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.862509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jswx\" (UniqueName: \"kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.864924 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.864899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.865056 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.864926    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.871415 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.871386    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5sp7\" (UniqueName: \"kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7\") pod \"authorino-6b5db55594-wg4m2\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") " pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:25.871545 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.871522    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jswx\" (UniqueName: \"kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx\") pod \"authorino-76877994-s7czc\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") " pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.895169 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.895132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-djbth" event={"ID":"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35","Type":"ContainerStarted","Data":"58ca385150447678d8af4abf0112ae6d336c0cf381033517b53e94596d56437a"}
Jul 24 19:43:25.895169 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.895153    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.901365 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.901343    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:25.963251 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.963164    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jswx\" (UniqueName: \"kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx\") pod \"f30c952f-6e80-4c93-8275-f87751f145d1\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") "
Jul 24 19:43:25.963251 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.963207    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert\") pod \"f30c952f-6e80-4c93-8275-f87751f145d1\" (UID: \"f30c952f-6e80-4c93-8275-f87751f145d1\") "
Jul 24 19:43:25.965284 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.965224    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx" (OuterVolumeSpecName: "kube-api-access-7jswx") pod "f30c952f-6e80-4c93-8275-f87751f145d1" (UID: "f30c952f-6e80-4c93-8275-f87751f145d1"). InnerVolumeSpecName "kube-api-access-7jswx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:43:25.965393 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:25.965323    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f30c952f-6e80-4c93-8275-f87751f145d1" (UID: "f30c952f-6e80-4c93-8275-f87751f145d1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:43:26.019581 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.019546    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:43:26.064079 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.064040    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f30c952f-6e80-4c93-8275-f87751f145d1-tls-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:26.064079 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.064075    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jswx\" (UniqueName: \"kubernetes.io/projected/f30c952f-6e80-4c93-8275-f87751f145d1-kube-api-access-7jswx\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:26.158020 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.153441    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:43:26.901615 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.901521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-djbth" event={"ID":"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35","Type":"ContainerStarted","Data":"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"}
Jul 24 19:43:26.901615 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.901547    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-djbth" podUID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" containerName="authorino" containerID="cri-o://4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47" gracePeriod=30
Jul 24 19:43:26.903470 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.903441    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5db55594-wg4m2" event={"ID":"c7d6b5b1-551e-4130-907c-abd39f8657a8","Type":"ContainerStarted","Data":"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"}
Jul 24 19:43:26.903605 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.903484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5db55594-wg4m2" event={"ID":"c7d6b5b1-551e-4130-907c-abd39f8657a8","Type":"ContainerStarted","Data":"93f5efa0955161fecd28df57874b630704d8fe14298a800630b45764ff3e10ef"}
Jul 24 19:43:26.903605 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.903548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-s7czc"
Jul 24 19:43:26.917505 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.917463    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-djbth" podStartSLOduration=1.454457148 podStartE2EDuration="1.917447594s" totalImagesPullingTime="462.990446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:43:26.915751763 +0000 UTC m=+538.010297376" watchObservedRunningTime="2026-07-24 19:43:26.917447594 +0000 UTC m=+538.011993207"
Jul 24 19:43:26.945607 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.945550    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b5db55594-wg4m2" podStartSLOduration=1.6011455300000002 podStartE2EDuration="1.945535832s" totalImagesPullingTime="344.390302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:43:26.944541286 +0000 UTC m=+538.039086900" watchObservedRunningTime="2026-07-24 19:43:26.945535832 +0000 UTC m=+538.040081445"
Jul 24 19:43:26.976399 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.976364    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-s7czc"]
Jul 24 19:43:26.983732 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:26.983693    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-s7czc"]
Jul 24 19:43:27.163472 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.163403    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:27.272640 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.272598    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn92k\" (UniqueName: \"kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k\") pod \"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35\" (UID: \"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35\") "
Jul 24 19:43:27.274682 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.274652    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k" (OuterVolumeSpecName: "kube-api-access-jn92k") pod "fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" (UID: "fe7303c9-bab1-49a8-bb03-5aaa0d3caa35"). InnerVolumeSpecName "kube-api-access-jn92k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:43:27.374003 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.373962    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn92k\" (UniqueName: \"kubernetes.io/projected/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35-kube-api-access-jn92k\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:27.430809 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.430731    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f30c952f-6e80-4c93-8275-f87751f145d1" path="/var/lib/kubelet/pods/f30c952f-6e80-4c93-8275-f87751f145d1/volumes"
Jul 24 19:43:27.909399 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.909368    2568 generic.go:356] "Generic (PLEG): container finished" podID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" containerID="4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47" exitCode=0
Jul 24 19:43:27.909835 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.909413    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-djbth"
Jul 24 19:43:27.909835 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.909456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-djbth" event={"ID":"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35","Type":"ContainerDied","Data":"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"}
Jul 24 19:43:27.909835 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.909502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-djbth" event={"ID":"fe7303c9-bab1-49a8-bb03-5aaa0d3caa35","Type":"ContainerDied","Data":"58ca385150447678d8af4abf0112ae6d336c0cf381033517b53e94596d56437a"}
Jul 24 19:43:27.909835 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.909523    2568 scope.go:117] "RemoveContainer" containerID="4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"
Jul 24 19:43:27.917907 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.917881    2568 scope.go:117] "RemoveContainer" containerID="4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"
Jul 24 19:43:27.918165 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:43:27.918145    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47\": container with ID starting with 4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47 not found: ID does not exist" containerID="4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"
Jul 24 19:43:27.918235 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.918172    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47"} err="failed to get container status \"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47\": rpc error: code = NotFound desc = could not find container \"4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47\": container with ID starting with 4a105f68c3acc0b5179d6055a3e3f886dd57ba0bf2a809afe9b4ef7d11b6ea47 not found: ID does not exist"
Jul 24 19:43:27.925965 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.925937    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:27.931337 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:27.931316    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-djbth"]
Jul 24 19:43:29.432053 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:29.432024    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" path="/var/lib/kubelet/pods/fe7303c9-bab1-49a8-bb03-5aaa0d3caa35/volumes"
Jul 24 19:43:31.589283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.589214    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:31.589806 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.589733    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" containerName="authorino"
Jul 24 19:43:31.589806 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.589754    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" containerName="authorino"
Jul 24 19:43:31.589918 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.589865    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7303c9-bab1-49a8-bb03-5aaa0d3caa35" containerName="authorino"
Jul 24 19:43:31.594327 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.594304    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.597300 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.597252    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 19:43:31.597417 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.597303    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dffmq\""
Jul 24 19:43:31.597473 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.597448    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 19:43:31.602789 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.602764    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:31.612616 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.612591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frdk\" (UniqueName: \"kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.612739 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.612690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.714048 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.714006    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.714224 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.714063    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frdk\" (UniqueName: \"kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.714224 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:43:31.714173    2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 24 19:43:31.714371 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:43:31.714254    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert podName:9ae116d2-0336-4e6f-a254-f8e3241c13e8 nodeName:}" failed. No retries permitted until 2026-07-24 19:43:32.214232418 +0000 UTC m=+543.308778013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert") pod "maas-controller-568994ddcf-27sdm" (UID: "9ae116d2-0336-4e6f-a254-f8e3241c13e8") : secret "maas-controller-webhook-cert" not found
Jul 24 19:43:31.722741 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.722711    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frdk\" (UniqueName: \"kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:31.749443 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.749396    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:43:31.753227 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.753210    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.759621 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.759593    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:43:31.815503 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.815467    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.815672 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.815546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkhwq\" (UniqueName: \"kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.916676 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.916581    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkhwq\" (UniqueName: \"kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.916676 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.916674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.919019 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.918995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:31.925521 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:31.925494    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkhwq\" (UniqueName: \"kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq\") pod \"maas-controller-5c9fc846b-mwh9m\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") " pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:32.065900 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.065862    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:32.204947 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.204916    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:43:32.207479 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:43:32.207450    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d2d9ed5_0ac1_4299_b281_a68230370782.slice/crio-487eaf5ee568e0e7f34ba4d5cb5a6894c91ee86394ed3e4db13c554e4934532f WatchSource:0}: Error finding container 487eaf5ee568e0e7f34ba4d5cb5a6894c91ee86394ed3e4db13c554e4934532f: Status 404 returned error can't find the container with id 487eaf5ee568e0e7f34ba4d5cb5a6894c91ee86394ed3e4db13c554e4934532f
Jul 24 19:43:32.219221 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.219189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:32.221761 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.221737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") pod \"maas-controller-568994ddcf-27sdm\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") " pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:32.506378 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.506285    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:32.635383 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.635356    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:32.637408 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:43:32.637381    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae116d2_0336_4e6f_a254_f8e3241c13e8.slice/crio-96d3caf848092e9162a22cac411f0869eeda3c0cc71711ed3d4e792504ba552f WatchSource:0}: Error finding container 96d3caf848092e9162a22cac411f0869eeda3c0cc71711ed3d4e792504ba552f: Status 404 returned error can't find the container with id 96d3caf848092e9162a22cac411f0869eeda3c0cc71711ed3d4e792504ba552f
Jul 24 19:43:32.937390 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.937232    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" event={"ID":"7d2d9ed5-0ac1-4299-b281-a68230370782","Type":"ContainerStarted","Data":"487eaf5ee568e0e7f34ba4d5cb5a6894c91ee86394ed3e4db13c554e4934532f"}
Jul 24 19:43:32.938590 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:32.938527    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568994ddcf-27sdm" event={"ID":"9ae116d2-0336-4e6f-a254-f8e3241c13e8","Type":"ContainerStarted","Data":"96d3caf848092e9162a22cac411f0869eeda3c0cc71711ed3d4e792504ba552f"}
Jul 24 19:43:34.951324 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:34.951291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" event={"ID":"7d2d9ed5-0ac1-4299-b281-a68230370782","Type":"ContainerStarted","Data":"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"}
Jul 24 19:43:34.951800 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:34.951385    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:34.952742 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:34.952718    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568994ddcf-27sdm" event={"ID":"9ae116d2-0336-4e6f-a254-f8e3241c13e8","Type":"ContainerStarted","Data":"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"}
Jul 24 19:43:34.952861 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:34.952756    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:34.986360 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:34.986312    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" podStartSLOduration=1.524984409 podStartE2EDuration="3.986296798s" totalImagesPullingTime="2.461312389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:43:34.983719436 +0000 UTC m=+546.078265048" watchObservedRunningTime="2026-07-24 19:43:34.986296798 +0000 UTC m=+546.080842436"
Jul 24 19:43:35.037895 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:35.037849    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-568994ddcf-27sdm" podStartSLOduration=1.999746993 podStartE2EDuration="4.037832198s" totalImagesPullingTime="2.038085205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:43:35.036962592 +0000 UTC m=+546.131508217" watchObservedRunningTime="2026-07-24 19:43:35.037832198 +0000 UTC m=+546.132377812"
Jul 24 19:43:45.959733 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:45.959697    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:45.960124 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:45.959772    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:43:46.024826 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.024786    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:46.025005 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.024983    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-568994ddcf-27sdm" podUID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" containerName="manager" containerID="cri-o://9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4" gracePeriod=30
Jul 24 19:43:46.273747 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.273723    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:46.347212 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.347177    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6frdk\" (UniqueName: \"kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk\") pod \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") "
Jul 24 19:43:46.347368 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.347228    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") pod \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\" (UID: \"9ae116d2-0336-4e6f-a254-f8e3241c13e8\") "
Jul 24 19:43:46.349454 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.349417    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9ae116d2-0336-4e6f-a254-f8e3241c13e8" (UID: "9ae116d2-0336-4e6f-a254-f8e3241c13e8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:43:46.349584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.349533    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk" (OuterVolumeSpecName: "kube-api-access-6frdk") pod "9ae116d2-0336-4e6f-a254-f8e3241c13e8" (UID: "9ae116d2-0336-4e6f-a254-f8e3241c13e8"). InnerVolumeSpecName "kube-api-access-6frdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:43:46.448370 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.448331    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6frdk\" (UniqueName: \"kubernetes.io/projected/9ae116d2-0336-4e6f-a254-f8e3241c13e8-kube-api-access-6frdk\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:46.448370 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:46.448363    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ae116d2-0336-4e6f-a254-f8e3241c13e8-webhook-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:43:47.013884 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.013852    2568 generic.go:356] "Generic (PLEG): container finished" podID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" containerID="9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4" exitCode=0
Jul 24 19:43:47.014396 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.013913    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-568994ddcf-27sdm"
Jul 24 19:43:47.014396 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.013942    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568994ddcf-27sdm" event={"ID":"9ae116d2-0336-4e6f-a254-f8e3241c13e8","Type":"ContainerDied","Data":"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"}
Jul 24 19:43:47.014396 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.014001    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-568994ddcf-27sdm" event={"ID":"9ae116d2-0336-4e6f-a254-f8e3241c13e8","Type":"ContainerDied","Data":"96d3caf848092e9162a22cac411f0869eeda3c0cc71711ed3d4e792504ba552f"}
Jul 24 19:43:47.014396 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.014024    2568 scope.go:117] "RemoveContainer" containerID="9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"
Jul 24 19:43:47.023248 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.023230    2568 scope.go:117] "RemoveContainer" containerID="9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"
Jul 24 19:43:47.023582 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:43:47.023564    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4\": container with ID starting with 9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4 not found: ID does not exist" containerID="9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"
Jul 24 19:43:47.023632 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.023590    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4"} err="failed to get container status \"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4\": rpc error: code = NotFound desc = could not find container \"9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4\": container with ID starting with 9b2e5f64fc8482f257fb9e326a6cb621cab3a1f064217719cc2a4a8286cfeae4 not found: ID does not exist"
Jul 24 19:43:47.035906 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.035880    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:47.040433 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.040413    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-568994ddcf-27sdm"]
Jul 24 19:43:47.431168 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:43:47.431086    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" path="/var/lib/kubelet/pods/9ae116d2-0336-4e6f-a254-f8e3241c13e8/volumes"
Jul 24 19:44:06.169187 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.169149    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-864cf74d58-49l72"]
Jul 24 19:44:06.169710 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.169512    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" containerName="manager"
Jul 24 19:44:06.169710 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.169524    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" containerName="manager"
Jul 24 19:44:06.169710 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.169591    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae116d2-0336-4e6f-a254-f8e3241c13e8" containerName="manager"
Jul 24 19:44:06.172696 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.172678    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.175566 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.175545    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 19:44:06.176999 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.176924    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nw9zh\""
Jul 24 19:44:06.182958 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.182932    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-864cf74d58-49l72"]
Jul 24 19:44:06.315290 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.315239    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-maas-api-tls\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.315467 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.315303    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-openshift-service-ca\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.315467 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.315325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9skmf\" (UniqueName: \"kubernetes.io/projected/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-kube-api-access-9skmf\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.416104 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.416072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-openshift-service-ca\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.416104 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.416110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9skmf\" (UniqueName: \"kubernetes.io/projected/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-kube-api-access-9skmf\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.416390 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.416182    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-maas-api-tls\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.416745 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.416725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-openshift-service-ca\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.418827 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.418797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-maas-api-tls\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.425101 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.425032    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9skmf\" (UniqueName: \"kubernetes.io/projected/509e08a6-35e6-43a1-9649-df8d5bf9ecdb-kube-api-access-9skmf\") pod \"maas-api-864cf74d58-49l72\" (UID: \"509e08a6-35e6-43a1-9649-df8d5bf9ecdb\") " pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.484902 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.484865    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:06.635734 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.635702    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-864cf74d58-49l72"]
Jul 24 19:44:06.802884 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.802847    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"]
Jul 24 19:44:06.807706 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.807682    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:06.810839 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.810814    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dzxmh\""
Jul 24 19:44:06.810982 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.810853    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 19:44:06.815229 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.815204    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"]
Jul 24 19:44:06.920912 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.920875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/634e271f-d2bf-40bc-ba90-828a063c25f8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:06.921073 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:06.920920    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4hjz\" (UniqueName: \"kubernetes.io/projected/634e271f-d2bf-40bc-ba90-828a063c25f8-kube-api-access-w4hjz\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.021665 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.021626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/634e271f-d2bf-40bc-ba90-828a063c25f8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.021665 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.021664    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4hjz\" (UniqueName: \"kubernetes.io/projected/634e271f-d2bf-40bc-ba90-828a063c25f8-kube-api-access-w4hjz\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.022413 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.022387    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/634e271f-d2bf-40bc-ba90-828a063c25f8-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.030941 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.030918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4hjz\" (UniqueName: \"kubernetes.io/projected/634e271f-d2bf-40bc-ba90-828a063c25f8-kube-api-access-w4hjz\") pod \"payload-pre-processing-687dbb6cf6-24tb9\" (UID: \"634e271f-d2bf-40bc-ba90-828a063c25f8\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.108458 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.108370    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72" event={"ID":"509e08a6-35e6-43a1-9649-df8d5bf9ecdb","Type":"ContainerStarted","Data":"f9e9c12372bbb1b9bdb067e3b01fcffcd87dea44144f7dfd72376cb7c951bc48"}
Jul 24 19:44:07.120616 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.120590    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:07.262291 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:07.262227    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"]
Jul 24 19:44:07.269457 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:44:07.268696    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634e271f_d2bf_40bc_ba90_828a063c25f8.slice/crio-95f6d82e5cdf87f023af1d04de021467c1582018e6cd139d9be65befe049847a WatchSource:0}: Error finding container 95f6d82e5cdf87f023af1d04de021467c1582018e6cd139d9be65befe049847a: Status 404 returned error can't find the container with id 95f6d82e5cdf87f023af1d04de021467c1582018e6cd139d9be65befe049847a
Jul 24 19:44:08.117207 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:08.117158    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9" event={"ID":"634e271f-d2bf-40bc-ba90-828a063c25f8","Type":"ContainerStarted","Data":"95f6d82e5cdf87f023af1d04de021467c1582018e6cd139d9be65befe049847a"}
Jul 24 19:44:09.125221 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:09.125164    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72" event={"ID":"509e08a6-35e6-43a1-9649-df8d5bf9ecdb","Type":"ContainerStarted","Data":"728aae08d6b1cff8431ccd3ea8eaa8021d4a44beb8b1d675390b4beea8f63525"}
Jul 24 19:44:09.125770 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:09.125373    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:09.141896 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:09.141840    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72" podStartSLOduration=1.494504917 podStartE2EDuration="3.141824676s" totalImagesPullingTime="1.647319759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:44:09.14047495 +0000 UTC m=+580.235020557" watchObservedRunningTime="2026-07-24 19:44:09.141824676 +0000 UTC m=+580.236370288"
Jul 24 19:44:10.130987 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:10.130945    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9" event={"ID":"634e271f-d2bf-40bc-ba90-828a063c25f8","Type":"ContainerStarted","Data":"8b434f1e61a0709235c423989fb0ee4652f34ae62a28dd37eeb36474c5a7c444"}
Jul 24 19:44:10.131479 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:10.131189    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:10.147537 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:10.147482    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9" podStartSLOduration=1.889360449 podStartE2EDuration="4.147467085s" totalImagesPullingTime="2.258106636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:44:10.1449481 +0000 UTC m=+581.239493713" watchObservedRunningTime="2026-07-24 19:44:10.147467085 +0000 UTC m=+581.242012698"
Jul 24 19:44:15.136809 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:15.136781    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-864cf74d58-49l72"
Jul 24 19:44:21.137771 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:21.137735    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-24tb9"
Jul 24 19:44:29.296111 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:29.296081    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:44:29.299633 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:29.299607    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:44:42.407307 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.407269    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"]
Jul 24 19:44:42.412535 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.412513    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.415151 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.415129    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 19:44:42.415151 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.415144    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x9sks\""
Jul 24 19:44:42.416706 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.416446    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 19:44:42.416706 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.416692    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 19:44:42.419863 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.419842    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"]
Jul 24 19:44:42.528776 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528747    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080695bd-b429-4e83-921a-504db28b663c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.528955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.528955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.528955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.529099 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528966    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfh9m\" (UniqueName: \"kubernetes.io/projected/080695bd-b429-4e83-921a-504db28b663c-kube-api-access-bfh9m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.529099 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.528985    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630327 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630340    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfh9m\" (UniqueName: \"kubernetes.io/projected/080695bd-b429-4e83-921a-504db28b663c-kube-api-access-bfh9m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080695bd-b429-4e83-921a-504db28b663c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630518 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630506    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630738 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630548    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630790 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630741    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630790 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.630955 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.630927    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.632716 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.632692    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080695bd-b429-4e83-921a-504db28b663c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.633019 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.633003    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080695bd-b429-4e83-921a-504db28b663c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.638192 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.638172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfh9m\" (UniqueName: \"kubernetes.io/projected/080695bd-b429-4e83-921a-504db28b663c-kube-api-access-bfh9m\") pod \"e2e-distinct-simulated-kserve-59975d69f9-l8cvl\" (UID: \"080695bd-b429-4e83-921a-504db28b663c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.725299 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.725247    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:42.859433 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:42.859347    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"]
Jul 24 19:44:42.861749 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:44:42.861716    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080695bd_b429_4e83_921a_504db28b663c.slice/crio-db2ee7d1db4fa8ec2b2c0b3c902d2190ed206605b9c61dfd2b9f9c77c7915930 WatchSource:0}: Error finding container db2ee7d1db4fa8ec2b2c0b3c902d2190ed206605b9c61dfd2b9f9c77c7915930: Status 404 returned error can't find the container with id db2ee7d1db4fa8ec2b2c0b3c902d2190ed206605b9c61dfd2b9f9c77c7915930
Jul 24 19:44:43.295316 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:43.295271    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl" event={"ID":"080695bd-b429-4e83-921a-504db28b663c","Type":"ContainerStarted","Data":"db2ee7d1db4fa8ec2b2c0b3c902d2190ed206605b9c61dfd2b9f9c77c7915930"}
Jul 24 19:44:44.302039 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:44.301988    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl" event={"ID":"080695bd-b429-4e83-921a-504db28b663c","Type":"ContainerStarted","Data":"69ef2b5dd51b85e18085f301692dd9a066dcc4f808f8fdb46db8de73e31ff2a1"}
Jul 24 19:44:44.321349 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:44.321299    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl" podStartSLOduration=1.297706674 podStartE2EDuration="2.321283191s" totalImagesPullingTime="1.023576517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:44:44.319483857 +0000 UTC m=+615.414029470" watchObservedRunningTime="2026-07-24 19:44:44.321283191 +0000 UTC m=+615.415828800"
Jul 24 19:44:52.725856 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:52.725764    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:52.725856 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:52.725852    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:52.737970 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:52.737943    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:44:53.355710 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:44:53.355676    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-l8cvl"
Jul 24 19:45:18.303152 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.303100    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"]
Jul 24 19:45:18.309943 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.309916    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.312583 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.312563    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 19:45:18.316743 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.316717    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"]
Jul 24 19:45:18.325995 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.325970    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.326098 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.326003    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fccd45-8d5f-494b-9761-33683ee72078-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.326098 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.326021    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.326098 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.326038    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj6td\" (UniqueName: \"kubernetes.io/projected/d1fccd45-8d5f-494b-9761-33683ee72078-kube-api-access-zj6td\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.326223 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.326103    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.326223 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.326135    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427008 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.426973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427008 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj6td\" (UniqueName: \"kubernetes.io/projected/d1fccd45-8d5f-494b-9761-33683ee72078-kube-api-access-zj6td\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427251 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427029    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427251 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427161    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427399 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427307    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427399 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fccd45-8d5f-494b-9761-33683ee72078-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427500 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427422    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427500 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427488    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.427665 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.427640    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.429274 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.429244    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1fccd45-8d5f-494b-9761-33683ee72078-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.429611 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.429594    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1fccd45-8d5f-494b-9761-33683ee72078-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.435028 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.435009    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj6td\" (UniqueName: \"kubernetes.io/projected/d1fccd45-8d5f-494b-9761-33683ee72078-kube-api-access-zj6td\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vx56r\" (UID: \"d1fccd45-8d5f-494b-9761-33683ee72078\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.622203 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.622121    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:18.762971 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.762943    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"]
Jul 24 19:45:18.765780 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:45:18.765752    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1fccd45_8d5f_494b_9761_33683ee72078.slice/crio-9c3517807415b89d28c9643300e290c3c6c092a116386f2b8185ce60cae68661 WatchSource:0}: Error finding container 9c3517807415b89d28c9643300e290c3c6c092a116386f2b8185ce60cae68661: Status 404 returned error can't find the container with id 9c3517807415b89d28c9643300e290c3c6c092a116386f2b8185ce60cae68661
Jul 24 19:45:18.767152 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:18.767133    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 19:45:19.466504 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:19.466468    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r" event={"ID":"d1fccd45-8d5f-494b-9761-33683ee72078","Type":"ContainerStarted","Data":"0185fceaf928f448e2b3b586548ad6f6ead14bf6fbd351932c22affdd89bffaf"}
Jul 24 19:45:19.466504 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:19.466509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r" event={"ID":"d1fccd45-8d5f-494b-9761-33683ee72078","Type":"ContainerStarted","Data":"9c3517807415b89d28c9643300e290c3c6c092a116386f2b8185ce60cae68661"}
Jul 24 19:45:19.484574 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:19.484531    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r" podStartSLOduration=1.291537769 podStartE2EDuration="1.484512478s" totalImagesPullingTime="192.974709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:45:19.482295006 +0000 UTC m=+650.576840620" watchObservedRunningTime="2026-07-24 19:45:19.484512478 +0000 UTC m=+650.579058092"
Jul 24 19:45:28.622597 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:28.622561    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:28.622970 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:28.622640    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:28.635090 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:28.635064    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:45:29.524520 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:45:29.524490    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vx56r"
Jul 24 19:46:37.795141 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:37.795054    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7779cf7f79-bbcbd"]
Jul 24 19:46:37.798960 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:37.798938    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:37.807480 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:37.807440    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7779cf7f79-bbcbd"]
Jul 24 19:46:37.923478 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:37.923437    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vbk\" (UniqueName: \"kubernetes.io/projected/1715dc07-fed9-4618-8ba0-8c126375bf8b-kube-api-access-c5vbk\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:37.923656 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:37.923485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1715dc07-fed9-4618-8ba0-8c126375bf8b-tls-cert\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.024734 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.024699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vbk\" (UniqueName: \"kubernetes.io/projected/1715dc07-fed9-4618-8ba0-8c126375bf8b-kube-api-access-c5vbk\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.024899 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.024741    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1715dc07-fed9-4618-8ba0-8c126375bf8b-tls-cert\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.027057 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.027029    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1715dc07-fed9-4618-8ba0-8c126375bf8b-tls-cert\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.034143 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.034118    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vbk\" (UniqueName: \"kubernetes.io/projected/1715dc07-fed9-4618-8ba0-8c126375bf8b-kube-api-access-c5vbk\") pod \"authorino-7779cf7f79-bbcbd\" (UID: \"1715dc07-fed9-4618-8ba0-8c126375bf8b\") " pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.108987 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.108894    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7779cf7f79-bbcbd"
Jul 24 19:46:38.253888 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.253861    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7779cf7f79-bbcbd"]
Jul 24 19:46:38.256448 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:46:38.256422    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1715dc07_fed9_4618_8ba0_8c126375bf8b.slice/crio-b190fe6b3b046104e6e6de8b1484953552aa58d13b2fb5b36faf9c944bc78324 WatchSource:0}: Error finding container b190fe6b3b046104e6e6de8b1484953552aa58d13b2fb5b36faf9c944bc78324: Status 404 returned error can't find the container with id b190fe6b3b046104e6e6de8b1484953552aa58d13b2fb5b36faf9c944bc78324
Jul 24 19:46:38.835691 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.835649    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7779cf7f79-bbcbd" event={"ID":"1715dc07-fed9-4618-8ba0-8c126375bf8b","Type":"ContainerStarted","Data":"adf0224d5e5134e8d6c10eec9b922a842619dcee75fd2a1efb01164b44096ff4"}
Jul 24 19:46:38.836156 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.835705    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7779cf7f79-bbcbd" event={"ID":"1715dc07-fed9-4618-8ba0-8c126375bf8b","Type":"ContainerStarted","Data":"b190fe6b3b046104e6e6de8b1484953552aa58d13b2fb5b36faf9c944bc78324"}
Jul 24 19:46:38.853978 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.853925    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7779cf7f79-bbcbd" podStartSLOduration=1.431589036 podStartE2EDuration="1.853909213s" totalImagesPullingTime="422.320177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:46:38.850809308 +0000 UTC m=+729.945354920" watchObservedRunningTime="2026-07-24 19:46:38.853909213 +0000 UTC m=+729.948454827"
Jul 24 19:46:38.879591 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.879548    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:46:38.879790 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:38.879764    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6b5db55594-wg4m2" podUID="c7d6b5b1-551e-4130-907c-abd39f8657a8" containerName="authorino" containerID="cri-o://911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f" gracePeriod=30
Jul 24 19:46:39.152079 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.152049    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:46:39.234370 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.234239    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5sp7\" (UniqueName: \"kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7\") pod \"c7d6b5b1-551e-4130-907c-abd39f8657a8\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") "
Jul 24 19:46:39.234533 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.234414    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert\") pod \"c7d6b5b1-551e-4130-907c-abd39f8657a8\" (UID: \"c7d6b5b1-551e-4130-907c-abd39f8657a8\") "
Jul 24 19:46:39.236494 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.236463    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7" (OuterVolumeSpecName: "kube-api-access-j5sp7") pod "c7d6b5b1-551e-4130-907c-abd39f8657a8" (UID: "c7d6b5b1-551e-4130-907c-abd39f8657a8"). InnerVolumeSpecName "kube-api-access-j5sp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:46:39.245429 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.245397    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c7d6b5b1-551e-4130-907c-abd39f8657a8" (UID: "c7d6b5b1-551e-4130-907c-abd39f8657a8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:46:39.336172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.336133    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5sp7\" (UniqueName: \"kubernetes.io/projected/c7d6b5b1-551e-4130-907c-abd39f8657a8-kube-api-access-j5sp7\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:46:39.336172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.336166    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c7d6b5b1-551e-4130-907c-abd39f8657a8-tls-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:46:39.841858 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.841817    2568 generic.go:356] "Generic (PLEG): container finished" podID="c7d6b5b1-551e-4130-907c-abd39f8657a8" containerID="911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f" exitCode=0
Jul 24 19:46:39.841858 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.841864    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5db55594-wg4m2"
Jul 24 19:46:39.842431 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.841904    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5db55594-wg4m2" event={"ID":"c7d6b5b1-551e-4130-907c-abd39f8657a8","Type":"ContainerDied","Data":"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"}
Jul 24 19:46:39.842431 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.841956    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5db55594-wg4m2" event={"ID":"c7d6b5b1-551e-4130-907c-abd39f8657a8","Type":"ContainerDied","Data":"93f5efa0955161fecd28df57874b630704d8fe14298a800630b45764ff3e10ef"}
Jul 24 19:46:39.842431 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.841981    2568 scope.go:117] "RemoveContainer" containerID="911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"
Jul 24 19:46:39.852438 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.852417    2568 scope.go:117] "RemoveContainer" containerID="911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"
Jul 24 19:46:39.852730 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:46:39.852710    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f\": container with ID starting with 911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f not found: ID does not exist" containerID="911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"
Jul 24 19:46:39.852808 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.852740    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f"} err="failed to get container status \"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f\": rpc error: code = NotFound desc = could not find container \"911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f\": container with ID starting with 911bd01bf50b5e438fbedc6bc9c1e735890881b3df1820eee4b831fcf268b44f not found: ID does not exist"
Jul 24 19:46:39.860803 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.860773    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:46:39.864340 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:39.864316    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b5db55594-wg4m2"]
Jul 24 19:46:41.436340 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:46:41.436244    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d6b5b1-551e-4130-907c-abd39f8657a8" path="/var/lib/kubelet/pods/c7d6b5b1-551e-4130-907c-abd39f8657a8/volumes"
Jul 24 19:47:25.234158 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.234117    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:47:25.234724 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.234369    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" podUID="7d2d9ed5-0ac1-4299-b281-a68230370782" containerName="manager" containerID="cri-o://855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478" gracePeriod=30
Jul 24 19:47:25.488836 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.488754    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:47:25.531837 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.531802    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert\") pod \"7d2d9ed5-0ac1-4299-b281-a68230370782\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") "
Jul 24 19:47:25.532016 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.531913    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkhwq\" (UniqueName: \"kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq\") pod \"7d2d9ed5-0ac1-4299-b281-a68230370782\" (UID: \"7d2d9ed5-0ac1-4299-b281-a68230370782\") "
Jul 24 19:47:25.533874 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.533841    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7d2d9ed5-0ac1-4299-b281-a68230370782" (UID: "7d2d9ed5-0ac1-4299-b281-a68230370782"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:47:25.533988 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.533904    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq" (OuterVolumeSpecName: "kube-api-access-pkhwq") pod "7d2d9ed5-0ac1-4299-b281-a68230370782" (UID: "7d2d9ed5-0ac1-4299-b281-a68230370782"). InnerVolumeSpecName "kube-api-access-pkhwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:47:25.633637 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.633601    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d2d9ed5-0ac1-4299-b281-a68230370782-webhook-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:47:25.633637 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:25.633635    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkhwq\" (UniqueName: \"kubernetes.io/projected/7d2d9ed5-0ac1-4299-b281-a68230370782-kube-api-access-pkhwq\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:47:26.074513 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.074475    2568 generic.go:356] "Generic (PLEG): container finished" podID="7d2d9ed5-0ac1-4299-b281-a68230370782" containerID="855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478" exitCode=0
Jul 24 19:47:26.074711 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.074524    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-mwh9m"
Jul 24 19:47:26.074711 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.074563    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" event={"ID":"7d2d9ed5-0ac1-4299-b281-a68230370782","Type":"ContainerDied","Data":"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"}
Jul 24 19:47:26.074711 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.074618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-mwh9m" event={"ID":"7d2d9ed5-0ac1-4299-b281-a68230370782","Type":"ContainerDied","Data":"487eaf5ee568e0e7f34ba4d5cb5a6894c91ee86394ed3e4db13c554e4934532f"}
Jul 24 19:47:26.074711 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.074641    2568 scope.go:117] "RemoveContainer" containerID="855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"
Jul 24 19:47:26.083356 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.083338    2568 scope.go:117] "RemoveContainer" containerID="855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"
Jul 24 19:47:26.083622 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:47:26.083604    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478\": container with ID starting with 855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478 not found: ID does not exist" containerID="855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"
Jul 24 19:47:26.083685 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.083632    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478"} err="failed to get container status \"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478\": rpc error: code = NotFound desc = could not find container \"855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478\": container with ID starting with 855278c21f882e88b82fdc4810e8558f5f92e6689d1345586579cce7b05af478 not found: ID does not exist"
Jul 24 19:47:26.096310 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.096285    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:47:26.099939 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:26.099919    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-mwh9m"]
Jul 24 19:47:27.431044 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:47:27.431009    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d2d9ed5-0ac1-4299-b281-a68230370782" path="/var/lib/kubelet/pods/7d2d9ed5-0ac1-4299-b281-a68230370782/volumes"
Jul 24 19:48:26.874930 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.874886    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:26.875516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875446    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7d6b5b1-551e-4130-907c-abd39f8657a8" containerName="authorino"
Jul 24 19:48:26.875516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875467    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d6b5b1-551e-4130-907c-abd39f8657a8" containerName="authorino"
Jul 24 19:48:26.875516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875504    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d2d9ed5-0ac1-4299-b281-a68230370782" containerName="manager"
Jul 24 19:48:26.875516 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875512    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d2d9ed5-0ac1-4299-b281-a68230370782" containerName="manager"
Jul 24 19:48:26.875739 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875610    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d2d9ed5-0ac1-4299-b281-a68230370782" containerName="manager"
Jul 24 19:48:26.875739 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.875625    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7d6b5b1-551e-4130-907c-abd39f8657a8" containerName="authorino"
Jul 24 19:48:26.878769 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.878748    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:26.882573 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.882548    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dffmq\""
Jul 24 19:48:26.882573 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.882551    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 19:48:26.882760 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.882551    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 19:48:26.886169 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.885934    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:26.988312 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.988248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:26.988477 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:26.988384    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x4xp\" (UniqueName: \"kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.089217 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.089174    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x4xp\" (UniqueName: \"kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.089413 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.089349    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.091948 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.091922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.098640 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.098611    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x4xp\" (UniqueName: \"kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp\") pod \"maas-controller-5c9fc846b-ppjxs\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") " pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.189914 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.189825    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:27.326731 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.326706    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:27.329069 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:48:27.329042    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcaf519_145f_43c4_b16e_ddae8b45789c.slice/crio-f9f0f5ebe9ab7ab9fefd774e368c15fd7e3c69c2136ae17c18ad3f3b935cd858 WatchSource:0}: Error finding container f9f0f5ebe9ab7ab9fefd774e368c15fd7e3c69c2136ae17c18ad3f3b935cd858: Status 404 returned error can't find the container with id f9f0f5ebe9ab7ab9fefd774e368c15fd7e3c69c2136ae17c18ad3f3b935cd858
Jul 24 19:48:27.363987 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:27.363951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" event={"ID":"5dcaf519-145f-43c4-b16e-ddae8b45789c","Type":"ContainerStarted","Data":"f9f0f5ebe9ab7ab9fefd774e368c15fd7e3c69c2136ae17c18ad3f3b935cd858"}
Jul 24 19:48:28.370841 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:28.370800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" event={"ID":"5dcaf519-145f-43c4-b16e-ddae8b45789c","Type":"ContainerStarted","Data":"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"}
Jul 24 19:48:28.371283 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:28.370869    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:28.391317 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:28.391252    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" podStartSLOduration=2.018492785 podStartE2EDuration="2.391234578s" totalImagesPullingTime="372.741793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:48:28.388896128 +0000 UTC m=+839.483441741" watchObservedRunningTime="2026-07-24 19:48:28.391234578 +0000 UTC m=+839.485780259"
Jul 24 19:48:39.379135 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:39.379099    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:44.020239 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.020192    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:44.020648 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.020462    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" podUID="5dcaf519-145f-43c4-b16e-ddae8b45789c" containerName="manager" containerID="cri-o://27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac" gracePeriod=30
Jul 24 19:48:44.276316 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.276244    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:44.337718 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.337688    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x4xp\" (UniqueName: \"kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp\") pod \"5dcaf519-145f-43c4-b16e-ddae8b45789c\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") "
Jul 24 19:48:44.337891 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.337725    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert\") pod \"5dcaf519-145f-43c4-b16e-ddae8b45789c\" (UID: \"5dcaf519-145f-43c4-b16e-ddae8b45789c\") "
Jul 24 19:48:44.339781 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.339751    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5dcaf519-145f-43c4-b16e-ddae8b45789c" (UID: "5dcaf519-145f-43c4-b16e-ddae8b45789c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 19:48:44.340104 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.340077    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp" (OuterVolumeSpecName: "kube-api-access-4x4xp") pod "5dcaf519-145f-43c4-b16e-ddae8b45789c" (UID: "5dcaf519-145f-43c4-b16e-ddae8b45789c"). InnerVolumeSpecName "kube-api-access-4x4xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:48:44.438614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.438577    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x4xp\" (UniqueName: \"kubernetes.io/projected/5dcaf519-145f-43c4-b16e-ddae8b45789c-kube-api-access-4x4xp\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:48:44.438614 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.438612    2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcaf519-145f-43c4-b16e-ddae8b45789c-webhook-cert\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:48:44.453021 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.452988    2568 generic.go:356] "Generic (PLEG): container finished" podID="5dcaf519-145f-43c4-b16e-ddae8b45789c" containerID="27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac" exitCode=0
Jul 24 19:48:44.453172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.453030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" event={"ID":"5dcaf519-145f-43c4-b16e-ddae8b45789c","Type":"ContainerDied","Data":"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"}
Jul 24 19:48:44.453172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.453042    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-ppjxs"
Jul 24 19:48:44.453172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.453069    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-ppjxs" event={"ID":"5dcaf519-145f-43c4-b16e-ddae8b45789c","Type":"ContainerDied","Data":"f9f0f5ebe9ab7ab9fefd774e368c15fd7e3c69c2136ae17c18ad3f3b935cd858"}
Jul 24 19:48:44.453172 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.453090    2568 scope.go:117] "RemoveContainer" containerID="27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"
Jul 24 19:48:44.464095 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.464078    2568 scope.go:117] "RemoveContainer" containerID="27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"
Jul 24 19:48:44.464393 ip-10-0-129-222 kubenswrapper[2568]: E0724 19:48:44.464373    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac\": container with ID starting with 27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac not found: ID does not exist" containerID="27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"
Jul 24 19:48:44.464453 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.464402    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac"} err="failed to get container status \"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac\": rpc error: code = NotFound desc = could not find container \"27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac\": container with ID starting with 27ed25f3cc1497f76a99273fc77695efb33c59335081dda193732c639cf80dac not found: ID does not exist"
Jul 24 19:48:44.475347 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.475322    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:44.478507 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:44.478486    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-ppjxs"]
Jul 24 19:48:45.252156 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.252113    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c9fc846b-4cqls"]
Jul 24 19:48:45.252748 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.252654    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dcaf519-145f-43c4-b16e-ddae8b45789c" containerName="manager"
Jul 24 19:48:45.252748 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.252674    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dcaf519-145f-43c4-b16e-ddae8b45789c" containerName="manager"
Jul 24 19:48:45.252748 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.252738    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dcaf519-145f-43c4-b16e-ddae8b45789c" containerName="manager"
Jul 24 19:48:45.257062 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.257044    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.260991 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.260970    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 19:48:45.261116 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.260972    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dffmq\""
Jul 24 19:48:45.261116 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.261028    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 19:48:45.266092 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.266069    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-4cqls"]
Jul 24 19:48:45.347627 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.347593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a263a1e2-8a11-449c-95c2-62789338ed07-webhook-cert\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.347797 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.347647    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqcb6\" (UniqueName: \"kubernetes.io/projected/a263a1e2-8a11-449c-95c2-62789338ed07-kube-api-access-cqcb6\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.430750 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.430718    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dcaf519-145f-43c4-b16e-ddae8b45789c" path="/var/lib/kubelet/pods/5dcaf519-145f-43c4-b16e-ddae8b45789c/volumes"
Jul 24 19:48:45.448361 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.448338    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a263a1e2-8a11-449c-95c2-62789338ed07-webhook-cert\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.448476 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.448384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqcb6\" (UniqueName: \"kubernetes.io/projected/a263a1e2-8a11-449c-95c2-62789338ed07-kube-api-access-cqcb6\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.450850 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.450817    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a263a1e2-8a11-449c-95c2-62789338ed07-webhook-cert\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.459849 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.459828    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqcb6\" (UniqueName: \"kubernetes.io/projected/a263a1e2-8a11-449c-95c2-62789338ed07-kube-api-access-cqcb6\") pod \"maas-controller-5c9fc846b-4cqls\" (UID: \"a263a1e2-8a11-449c-95c2-62789338ed07\") " pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.567927 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.567839    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:45.703456 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:45.703426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c9fc846b-4cqls"]
Jul 24 19:48:45.705550 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:48:45.705520    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda263a1e2_8a11_449c_95c2_62789338ed07.slice/crio-88d7286aa530084739453726c498659444170027bef9313a9607ec3ab4f1e6b6 WatchSource:0}: Error finding container 88d7286aa530084739453726c498659444170027bef9313a9607ec3ab4f1e6b6: Status 404 returned error can't find the container with id 88d7286aa530084739453726c498659444170027bef9313a9607ec3ab4f1e6b6
Jul 24 19:48:46.467091 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:46.467055    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-4cqls" event={"ID":"a263a1e2-8a11-449c-95c2-62789338ed07","Type":"ContainerStarted","Data":"ca67b6015f3d95bc98edbca542ac7da0b3cdeca43e299b3e143073c8bd9bd464"}
Jul 24 19:48:46.467511 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:46.467206    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:48:46.467511 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:46.467249    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c9fc846b-4cqls" event={"ID":"a263a1e2-8a11-449c-95c2-62789338ed07","Type":"ContainerStarted","Data":"88d7286aa530084739453726c498659444170027bef9313a9607ec3ab4f1e6b6"}
Jul 24 19:48:46.484607 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:46.484561    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c9fc846b-4cqls" podStartSLOduration=1.178476978 podStartE2EDuration="1.484545693s" totalImagesPullingTime="306.068715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:48:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:48:46.48268483 +0000 UTC m=+857.577230444" watchObservedRunningTime="2026-07-24 19:48:46.484545693 +0000 UTC m=+857.579091340"
Jul 24 19:48:57.474522 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:48:57.474490    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c9fc846b-4cqls"
Jul 24 19:49:29.325313 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:49:29.325229    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:49:29.325313 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:49:29.325229    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:53:32.303052 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:32.303016    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:53:32.303658 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:32.303227    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" podUID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" containerName="manager" containerID="cri-o://ab7b73528edfb665933125b3f499819275e54037dd73395e4c6bb04d3f93e13e" gracePeriod=10
Jul 24 19:53:35.868660 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:35.868628    2568 generic.go:356] "Generic (PLEG): container finished" podID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" containerID="ab7b73528edfb665933125b3f499819275e54037dd73395e4c6bb04d3f93e13e" exitCode=0
Jul 24 19:53:35.869027 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:35.868692    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" event={"ID":"6a9fbbf7-7fd6-42b5-9078-2d555659b617","Type":"ContainerDied","Data":"ab7b73528edfb665933125b3f499819275e54037dd73395e4c6bb04d3f93e13e"}
Jul 24 19:53:35.962945 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:35.962920    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:53:36.052579 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.052489    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-529np\" (UniqueName: \"kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np\") pod \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") "
Jul 24 19:53:36.052579 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.052556    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume\") pod \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\" (UID: \"6a9fbbf7-7fd6-42b5-9078-2d555659b617\") "
Jul 24 19:53:36.052954 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.052927    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a9fbbf7-7fd6-42b5-9078-2d555659b617" (UID: "6a9fbbf7-7fd6-42b5-9078-2d555659b617"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 19:53:36.054680 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.054660    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np" (OuterVolumeSpecName: "kube-api-access-529np") pod "6a9fbbf7-7fd6-42b5-9078-2d555659b617" (UID: "6a9fbbf7-7fd6-42b5-9078-2d555659b617"). InnerVolumeSpecName "kube-api-access-529np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 19:53:36.154187 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.154153    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-529np\" (UniqueName: \"kubernetes.io/projected/6a9fbbf7-7fd6-42b5-9078-2d555659b617-kube-api-access-529np\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:53:36.154187 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.154182    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a9fbbf7-7fd6-42b5-9078-2d555659b617-extensions-socket-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 19:53:36.875103 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.875074    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"
Jul 24 19:53:36.875103 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.875086    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc" event={"ID":"6a9fbbf7-7fd6-42b5-9078-2d555659b617","Type":"ContainerDied","Data":"3c1fc6c0843ba49c085f46ba65842e9a960370632d6abb2c0242411afceea11c"}
Jul 24 19:53:36.875555 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.875149    2568 scope.go:117] "RemoveContainer" containerID="ab7b73528edfb665933125b3f499819275e54037dd73395e4c6bb04d3f93e13e"
Jul 24 19:53:36.898566 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.898536    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:53:36.903602 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:36.903576    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wdxlc"]
Jul 24 19:53:37.431202 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:53:37.431158    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" path="/var/lib/kubelet/pods/6a9fbbf7-7fd6-42b5-9078-2d555659b617/volumes"
Jul 24 19:54:29.360584 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:29.360552    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:54:29.362531 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:29.362507    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:54:33.432896 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.432861    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"]
Jul 24 19:54:33.433282 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.433226    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" containerName="manager"
Jul 24 19:54:33.433282 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.433237    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" containerName="manager"
Jul 24 19:54:33.433365 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.433322    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a9fbbf7-7fd6-42b5-9078-2d555659b617" containerName="manager"
Jul 24 19:54:33.437445 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.437427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.440077 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.440055    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dztm7\""
Jul 24 19:54:33.447701 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.447675    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"]
Jul 24 19:54:33.450451 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.450430    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686afa03-4122-4c5c-89f3-0baa54f57126-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.450561 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.450520    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7tmc\" (UniqueName: \"kubernetes.io/projected/686afa03-4122-4c5c-89f3-0baa54f57126-kube-api-access-j7tmc\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.551690 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.551655    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7tmc\" (UniqueName: \"kubernetes.io/projected/686afa03-4122-4c5c-89f3-0baa54f57126-kube-api-access-j7tmc\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.551854 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.551748    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686afa03-4122-4c5c-89f3-0baa54f57126-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.552177 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.552153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/686afa03-4122-4c5c-89f3-0baa54f57126-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.562016 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.561985    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7tmc\" (UniqueName: \"kubernetes.io/projected/686afa03-4122-4c5c-89f3-0baa54f57126-kube-api-access-j7tmc\") pod \"kuadrant-operator-controller-manager-86878c769f-mz8px\" (UID: \"686afa03-4122-4c5c-89f3-0baa54f57126\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.749092 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.749059    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:33.891295 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:33.891253    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"]
Jul 24 19:54:33.894198 ip-10-0-129-222 kubenswrapper[2568]: W0724 19:54:33.894168    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686afa03_4122_4c5c_89f3_0baa54f57126.slice/crio-94b3719ee829e3bfcbda9a2fb99e79a69d323b7f0bfa69501dd0e071becc24de WatchSource:0}: Error finding container 94b3719ee829e3bfcbda9a2fb99e79a69d323b7f0bfa69501dd0e071becc24de: Status 404 returned error can't find the container with id 94b3719ee829e3bfcbda9a2fb99e79a69d323b7f0bfa69501dd0e071becc24de
Jul 24 19:54:34.149423 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:34.149331    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px" event={"ID":"686afa03-4122-4c5c-89f3-0baa54f57126","Type":"ContainerStarted","Data":"3cdfdf9bc8958f3e2599e6e7e767df68c8539609d61b54612a78c4f902803922"}
Jul 24 19:54:34.149423 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:34.149392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px" event={"ID":"686afa03-4122-4c5c-89f3-0baa54f57126","Type":"ContainerStarted","Data":"94b3719ee829e3bfcbda9a2fb99e79a69d323b7f0bfa69501dd0e071becc24de"}
Jul 24 19:54:34.149633 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:34.149497    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:54:34.171694 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:34.171645    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px" podStartSLOduration=1.171627651 podStartE2EDuration="1.171627651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 19:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 19:54:34.169642273 +0000 UTC m=+1205.264187886" watchObservedRunningTime="2026-07-24 19:54:34.171627651 +0000 UTC m=+1205.266173264"
Jul 24 19:54:45.157202 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:54:45.157171    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mz8px"
Jul 24 19:59:29.398704 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:59:29.398673    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 19:59:29.402787 ip-10-0-129-222 kubenswrapper[2568]: I0724 19:59:29.402765    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:01:27.812173 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.812077    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:01:27.816417 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.816397    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:27.819115 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.819091    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-844e866c-serving-cert\""
Jul 24 20:01:27.822967 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.822910    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:01:27.929649 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.929606    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:27.929843 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.929686    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:27.929843 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:27.929787    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zx4\" (UniqueName: \"kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.030977 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.030926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zx4\" (UniqueName: \"kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.031175 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.030995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.031175 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.031043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.031175 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:01:28.031147    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-serving-cert: secret "maas-api-e2e-ait-844e866c-serving-cert" not found
Jul 24 20:01:28.031377 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:01:28.031226    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls podName:07358d62-5eaf-4253-bbdd-0a353cc65c48 nodeName:}" failed. No retries permitted until 2026-07-24 20:01:28.531206452 +0000 UTC m=+1619.625752063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls") pod "maas-api-e2e-ait-844e866c-868c877699-b8kj8" (UID: "07358d62-5eaf-4253-bbdd-0a353cc65c48") : secret "maas-api-e2e-ait-844e866c-serving-cert" not found
Jul 24 20:01:28.031782 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.031761    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.042082 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.042054    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zx4\" (UniqueName: \"kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.479592 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.479548    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:01:28.483157 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.483140    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.485764 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.485742    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-844e866c\""
Jul 24 20:01:28.494490 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.494465    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:01:28.519642 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.519612    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:01:28.523012 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.522990    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.525850 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.525832    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-844e866c-dockercfg-gvbfc\""
Jul 24 20:01:28.532761 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.532736    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:01:28.534494 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.534471    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.534608 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.534564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5crj\" (UniqueName: \"kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.534728 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.534699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.537224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.537202    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") pod \"maas-api-e2e-ait-844e866c-868c877699-b8kj8\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.635227 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.635197    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5crj\" (UniqueName: \"kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.635410 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.635248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.635410 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.635307    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.635410 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.635333    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbxj4\" (UniqueName: \"kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.636005 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.635986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.643083 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.643053    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5crj\" (UniqueName: \"kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj\") pod \"payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.730150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.730061    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:28.736653 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.736615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbxj4\" (UniqueName: \"kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.736805 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.736786    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.737446 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.737427    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.744427 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.744406    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbxj4\" (UniqueName: \"kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4\") pod \"payload-processing-e2e-ait-844e866c-587d66d88-4cw8c\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") " pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.794635 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.794599    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:28.834480 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.834448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:28.885389 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.885359    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:01:28.887426 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:01:28.887398    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07358d62_5eaf_4253_bbdd_0a353cc65c48.slice/crio-6955bedab586247e34ca12fdf866016cda68705b270058d296878c10e25ec02c WatchSource:0}: Error finding container 6955bedab586247e34ca12fdf866016cda68705b270058d296878c10e25ec02c: Status 404 returned error can't find the container with id 6955bedab586247e34ca12fdf866016cda68705b270058d296878c10e25ec02c
Jul 24 20:01:28.888800 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:28.888779    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:01:29.156746 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:29.156654    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" event={"ID":"07358d62-5eaf-4253-bbdd-0a353cc65c48","Type":"ContainerStarted","Data":"6955bedab586247e34ca12fdf866016cda68705b270058d296878c10e25ec02c"}
Jul 24 20:01:29.170793 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:29.170767    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:01:29.172577 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:01:29.172553    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd45741_be89_4979_bd60_5310e548a191.slice/crio-d609d8fbe5654b740f37677c5eb9b2716bb9d01003e51ea5a815435fda8ee560 WatchSource:0}: Error finding container d609d8fbe5654b740f37677c5eb9b2716bb9d01003e51ea5a815435fda8ee560: Status 404 returned error can't find the container with id d609d8fbe5654b740f37677c5eb9b2716bb9d01003e51ea5a815435fda8ee560
Jul 24 20:01:29.205491 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:29.205467    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:01:29.217470 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:01:29.217437    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139ced60_1443_4b9d_b364_eea11a6c3582.slice/crio-7d4202d26a94eb8e10b26ba0933cea52ea31d21f95ed04c86f85342ad275984e WatchSource:0}: Error finding container 7d4202d26a94eb8e10b26ba0933cea52ea31d21f95ed04c86f85342ad275984e: Status 404 returned error can't find the container with id 7d4202d26a94eb8e10b26ba0933cea52ea31d21f95ed04c86f85342ad275984e
Jul 24 20:01:30.164455 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.164418    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" event={"ID":"07358d62-5eaf-4253-bbdd-0a353cc65c48","Type":"ContainerStarted","Data":"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"}
Jul 24 20:01:30.164912 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.164491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:30.166656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.166630    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" event={"ID":"139ced60-1443-4b9d-b364-eea11a6c3582","Type":"ContainerStarted","Data":"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"}
Jul 24 20:01:30.166794 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.166677    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" event={"ID":"139ced60-1443-4b9d-b364-eea11a6c3582","Type":"ContainerStarted","Data":"7d4202d26a94eb8e10b26ba0933cea52ea31d21f95ed04c86f85342ad275984e"}
Jul 24 20:01:30.168607 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.168581    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" event={"ID":"efd45741-be89-4979-bd60-5310e548a191","Type":"ContainerStarted","Data":"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"}
Jul 24 20:01:30.168734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.168631    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" event={"ID":"efd45741-be89-4979-bd60-5310e548a191","Type":"ContainerStarted","Data":"d609d8fbe5654b740f37677c5eb9b2716bb9d01003e51ea5a815435fda8ee560"}
Jul 24 20:01:30.168734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.168675    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:30.182120 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.182078    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" podStartSLOduration=2.560467161 podStartE2EDuration="3.182063998s" totalImagesPullingTime="621.596837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:01:30.180003044 +0000 UTC m=+1621.274548659" watchObservedRunningTime="2026-07-24 20:01:30.182063998 +0000 UTC m=+1621.276609611"
Jul 24 20:01:30.195603 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.195559    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" podStartSLOduration=2.195538481 podStartE2EDuration="2.195538481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:01:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:01:30.193596513 +0000 UTC m=+1621.288142127" watchObservedRunningTime="2026-07-24 20:01:30.195538481 +0000 UTC m=+1621.290084094"
Jul 24 20:01:30.210196 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:30.210153    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" podStartSLOduration=2.210132243 podStartE2EDuration="2.210132243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:01:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:01:30.208333614 +0000 UTC m=+1621.302879227" watchObservedRunningTime="2026-07-24 20:01:30.210132243 +0000 UTC m=+1621.304677856"
Jul 24 20:01:31.180461 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:31.180411    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:01:36.186082 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:36.186044    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:01:41.182914 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:41.182878    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:01:42.187247 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:01:42.187216    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:02:03.743720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.743682    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:02:03.744365 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.744033    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" podUID="07358d62-5eaf-4253-bbdd-0a353cc65c48" containerName="maas-api" containerID="cri-o://f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6" gracePeriod=30
Jul 24 20:02:03.804074 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.804034    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:02:03.804438 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.804312    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" podUID="139ced60-1443-4b9d-b364-eea11a6c3582" containerName="payload-processing" containerID="cri-o://7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313" gracePeriod=30
Jul 24 20:02:03.811909 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.811875    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:02:03.812080 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:03.812056    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" podUID="efd45741-be89-4979-bd60-5310e548a191" containerName="payload-pre-processing" containerID="cri-o://eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa" gracePeriod=30
Jul 24 20:02:04.085905 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.085869    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:02:04.088775 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.088754    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:02:04.116647 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.116614    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:02:04.165132 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165101    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") pod \"07358d62-5eaf-4253-bbdd-0a353cc65c48\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") "
Jul 24 20:02:04.165317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165153    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca\") pod \"07358d62-5eaf-4253-bbdd-0a353cc65c48\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") "
Jul 24 20:02:04.165317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165176    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume\") pod \"efd45741-be89-4979-bd60-5310e548a191\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") "
Jul 24 20:02:04.165317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165199    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume\") pod \"139ced60-1443-4b9d-b364-eea11a6c3582\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") "
Jul 24 20:02:04.165317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165274    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9zx4\" (UniqueName: \"kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4\") pod \"07358d62-5eaf-4253-bbdd-0a353cc65c48\" (UID: \"07358d62-5eaf-4253-bbdd-0a353cc65c48\") "
Jul 24 20:02:04.165519 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165377    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5crj\" (UniqueName: \"kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj\") pod \"efd45741-be89-4979-bd60-5310e548a191\" (UID: \"efd45741-be89-4979-bd60-5310e548a191\") "
Jul 24 20:02:04.165519 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbxj4\" (UniqueName: \"kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4\") pod \"139ced60-1443-4b9d-b364-eea11a6c3582\" (UID: \"139ced60-1443-4b9d-b364-eea11a6c3582\") "
Jul 24 20:02:04.165637 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165551    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "07358d62-5eaf-4253-bbdd-0a353cc65c48" (UID: "07358d62-5eaf-4253-bbdd-0a353cc65c48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:04.165637 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165577    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efd45741-be89-4979-bd60-5310e548a191" (UID: "efd45741-be89-4979-bd60-5310e548a191"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:04.165807 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165724    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07358d62-5eaf-4253-bbdd-0a353cc65c48-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.165807 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165746    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd45741-be89-4979-bd60-5310e548a191-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.165903 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.165819    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "139ced60-1443-4b9d-b364-eea11a6c3582" (UID: "139ced60-1443-4b9d-b364-eea11a6c3582"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:04.167549 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.167520    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "07358d62-5eaf-4253-bbdd-0a353cc65c48" (UID: "07358d62-5eaf-4253-bbdd-0a353cc65c48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:02:04.167838 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.167809    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4" (OuterVolumeSpecName: "kube-api-access-j9zx4") pod "07358d62-5eaf-4253-bbdd-0a353cc65c48" (UID: "07358d62-5eaf-4253-bbdd-0a353cc65c48"). InnerVolumeSpecName "kube-api-access-j9zx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:04.167992 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.167976    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4" (OuterVolumeSpecName: "kube-api-access-pbxj4") pod "139ced60-1443-4b9d-b364-eea11a6c3582" (UID: "139ced60-1443-4b9d-b364-eea11a6c3582"). InnerVolumeSpecName "kube-api-access-pbxj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:04.168041 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.167999    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj" (OuterVolumeSpecName: "kube-api-access-c5crj") pod "efd45741-be89-4979-bd60-5310e548a191" (UID: "efd45741-be89-4979-bd60-5310e548a191"). InnerVolumeSpecName "kube-api-access-c5crj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:04.266661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.266580    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07358d62-5eaf-4253-bbdd-0a353cc65c48-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.266661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.266610    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/139ced60-1443-4b9d-b364-eea11a6c3582-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.266661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.266626    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9zx4\" (UniqueName: \"kubernetes.io/projected/07358d62-5eaf-4253-bbdd-0a353cc65c48-kube-api-access-j9zx4\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.266661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.266637    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5crj\" (UniqueName: \"kubernetes.io/projected/efd45741-be89-4979-bd60-5310e548a191-kube-api-access-c5crj\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.266661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.266646    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbxj4\" (UniqueName: \"kubernetes.io/projected/139ced60-1443-4b9d-b364-eea11a6c3582-kube-api-access-pbxj4\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:04.355495 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.355459    2568 generic.go:356] "Generic (PLEG): container finished" podID="07358d62-5eaf-4253-bbdd-0a353cc65c48" containerID="f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6" exitCode=0
Jul 24 20:02:04.355699 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.355528    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"
Jul 24 20:02:04.355699 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.355560    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" event={"ID":"07358d62-5eaf-4253-bbdd-0a353cc65c48","Type":"ContainerDied","Data":"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"}
Jul 24 20:02:04.355699 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.355617    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8" event={"ID":"07358d62-5eaf-4253-bbdd-0a353cc65c48","Type":"ContainerDied","Data":"6955bedab586247e34ca12fdf866016cda68705b270058d296878c10e25ec02c"}
Jul 24 20:02:04.355699 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.355642    2568 scope.go:117] "RemoveContainer" containerID="f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"
Jul 24 20:02:04.357239 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.357211    2568 generic.go:356] "Generic (PLEG): container finished" podID="139ced60-1443-4b9d-b364-eea11a6c3582" containerID="7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313" exitCode=0
Jul 24 20:02:04.357392 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.357284    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"
Jul 24 20:02:04.357392 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.357329    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" event={"ID":"139ced60-1443-4b9d-b364-eea11a6c3582","Type":"ContainerDied","Data":"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"}
Jul 24 20:02:04.357392 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.357374    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c" event={"ID":"139ced60-1443-4b9d-b364-eea11a6c3582","Type":"ContainerDied","Data":"7d4202d26a94eb8e10b26ba0933cea52ea31d21f95ed04c86f85342ad275984e"}
Jul 24 20:02:04.358779 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.358738    2568 generic.go:356] "Generic (PLEG): container finished" podID="efd45741-be89-4979-bd60-5310e548a191" containerID="eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa" exitCode=0
Jul 24 20:02:04.358779 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.358777    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"
Jul 24 20:02:04.358956 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.358837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" event={"ID":"efd45741-be89-4979-bd60-5310e548a191","Type":"ContainerDied","Data":"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"}
Jul 24 20:02:04.358956 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.358909    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm" event={"ID":"efd45741-be89-4979-bd60-5310e548a191","Type":"ContainerDied","Data":"d609d8fbe5654b740f37677c5eb9b2716bb9d01003e51ea5a815435fda8ee560"}
Jul 24 20:02:04.366111 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.366012    2568 scope.go:117] "RemoveContainer" containerID="f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"
Jul 24 20:02:04.366792 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:04.366707    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6\": container with ID starting with f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6 not found: ID does not exist" containerID="f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"
Jul 24 20:02:04.366792 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.366745    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6"} err="failed to get container status \"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6\": rpc error: code = NotFound desc = could not find container \"f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6\": container with ID starting with f023a211d1e148a4df528daba06f3e45c3c84a18b21b8a6590cfb5685313cbc6 not found: ID does not exist"
Jul 24 20:02:04.366792 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.366769    2568 scope.go:117] "RemoveContainer" containerID="7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"
Jul 24 20:02:04.375908 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.375893    2568 scope.go:117] "RemoveContainer" containerID="7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"
Jul 24 20:02:04.376153 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:04.376136    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313\": container with ID starting with 7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313 not found: ID does not exist" containerID="7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"
Jul 24 20:02:04.376193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.376160    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313"} err="failed to get container status \"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313\": rpc error: code = NotFound desc = could not find container \"7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313\": container with ID starting with 7b6356a464b362834ea01cfd57856dbd963aaceba67d3123befaa103ff8e3313 not found: ID does not exist"
Jul 24 20:02:04.376193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.376177    2568 scope.go:117] "RemoveContainer" containerID="eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"
Jul 24 20:02:04.380729 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.380707    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:02:04.383974 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.383955    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-844e866c-868c877699-b8kj8"]
Jul 24 20:02:04.384164 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.384146    2568 scope.go:117] "RemoveContainer" containerID="eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"
Jul 24 20:02:04.384475 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:04.384452    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa\": container with ID starting with eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa not found: ID does not exist" containerID="eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"
Jul 24 20:02:04.384532 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.384481    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa"} err="failed to get container status \"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa\": rpc error: code = NotFound desc = could not find container \"eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa\": container with ID starting with eef9b0b3c5cb84d5a58b75ddb1e7821440afb65739409a7cb8109aa4c5d7f1fa not found: ID does not exist"
Jul 24 20:02:04.394201 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.394177    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:02:04.397493 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.397473    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-844e866c-7b674f48dd-hn4bm"]
Jul 24 20:02:04.409442 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.409396    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:02:04.411079 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:04.411063    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-844e866c-587d66d88-4cw8c"]
Jul 24 20:02:05.430302 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:05.430240    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07358d62-5eaf-4253-bbdd-0a353cc65c48" path="/var/lib/kubelet/pods/07358d62-5eaf-4253-bbdd-0a353cc65c48/volumes"
Jul 24 20:02:05.430643 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:05.430609    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139ced60-1443-4b9d-b364-eea11a6c3582" path="/var/lib/kubelet/pods/139ced60-1443-4b9d-b364-eea11a6c3582/volumes"
Jul 24 20:02:05.430922 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:05.430908    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efd45741-be89-4979-bd60-5310e548a191" path="/var/lib/kubelet/pods/efd45741-be89-4979-bd60-5310e548a191/volumes"
Jul 24 20:02:12.132477 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.132444    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:12.133175 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133147    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07358d62-5eaf-4253-bbdd-0a353cc65c48" containerName="maas-api"
Jul 24 20:02:12.133310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133178    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="07358d62-5eaf-4253-bbdd-0a353cc65c48" containerName="maas-api"
Jul 24 20:02:12.133310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133195    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139ced60-1443-4b9d-b364-eea11a6c3582" containerName="payload-processing"
Jul 24 20:02:12.133310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133204    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="139ced60-1443-4b9d-b364-eea11a6c3582" containerName="payload-processing"
Jul 24 20:02:12.133310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133226    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efd45741-be89-4979-bd60-5310e548a191" containerName="payload-pre-processing"
Jul 24 20:02:12.133310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133232    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="efd45741-be89-4979-bd60-5310e548a191" containerName="payload-pre-processing"
Jul 24 20:02:12.133574 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133337    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="efd45741-be89-4979-bd60-5310e548a191" containerName="payload-pre-processing"
Jul 24 20:02:12.133574 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133349    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="07358d62-5eaf-4253-bbdd-0a353cc65c48" containerName="maas-api"
Jul 24 20:02:12.133574 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.133364    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="139ced60-1443-4b9d-b364-eea11a6c3582" containerName="payload-processing"
Jul 24 20:02:12.140373 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.140352    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.142924 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.142903    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-bc219aa6-serving-cert\""
Jul 24 20:02:12.148504 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.148165    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:12.232971 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.232935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.233147 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.233023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft6g2\" (UniqueName: \"kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.233147 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.233053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.334224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.334192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft6g2\" (UniqueName: \"kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.334431 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.334230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.334431 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.334294    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.334889 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.334871    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.336760 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.336740    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.342486 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.342465    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft6g2\" (UniqueName: \"kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2\") pod \"maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.454187 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.454132    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:12.599071 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.598996    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:12.601756 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:12.601728    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444c6353_04de_477f_9288_9435c39a16ea.slice/crio-7e4439d7a747ae821e94b9bb594c59c6ff869ac56ef97f99aa497b8b4653a1e9 WatchSource:0}: Error finding container 7e4439d7a747ae821e94b9bb594c59c6ff869ac56ef97f99aa497b8b4653a1e9: Status 404 returned error can't find the container with id 7e4439d7a747ae821e94b9bb594c59c6ff869ac56ef97f99aa497b8b4653a1e9
Jul 24 20:02:12.751785 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.751710    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:12.756726 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.756699    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.759402 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.759375    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-bc219aa6\""
Jul 24 20:02:12.766943 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.766915    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:12.786453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.786413    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:12.791425 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.791400    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.793995 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.793968    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-bc219aa6-dockercfg-fxstp\""
Jul 24 20:02:12.798312 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.798287    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:12.838740 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.838708    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4kwj\" (UniqueName: \"kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.838907 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.838745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.838907 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.838777    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtv77\" (UniqueName: \"kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.838907 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.838831    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.939820 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.939782    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4kwj\" (UniqueName: \"kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.939820 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.939825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.940025 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.939865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtv77\" (UniqueName: \"kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.940025 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.939891    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.940831 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.940807    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:12.940877 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.940807    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.948731 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.948708    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtv77\" (UniqueName: \"kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77\") pod \"payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:12.948731 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:12.948720    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4kwj\" (UniqueName: \"kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj\") pod \"payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") " pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:13.071011 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.070922    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:13.104171 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.104136    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:13.264501 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.264466    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:13.268444 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:13.268411    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca22a32_f95b_4537_b497_6cfa08dc3634.slice/crio-4cc993931b082f6edbd1459177201e7edf169b8c42c4d0a616a534be4a29dd4a WatchSource:0}: Error finding container 4cc993931b082f6edbd1459177201e7edf169b8c42c4d0a616a534be4a29dd4a: Status 404 returned error can't find the container with id 4cc993931b082f6edbd1459177201e7edf169b8c42c4d0a616a534be4a29dd4a
Jul 24 20:02:13.411331 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.411297    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" event={"ID":"fca22a32-f95b-4537-b497-6cfa08dc3634","Type":"ContainerStarted","Data":"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"}
Jul 24 20:02:13.411527 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.411352    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" event={"ID":"fca22a32-f95b-4537-b497-6cfa08dc3634","Type":"ContainerStarted","Data":"4cc993931b082f6edbd1459177201e7edf169b8c42c4d0a616a534be4a29dd4a"}
Jul 24 20:02:13.411527 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.411380    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:13.413324 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.413294    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" event={"ID":"444c6353-04de-477f-9288-9435c39a16ea","Type":"ContainerStarted","Data":"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"}
Jul 24 20:02:13.413465 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.413341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" event={"ID":"444c6353-04de-477f-9288-9435c39a16ea","Type":"ContainerStarted","Data":"7e4439d7a747ae821e94b9bb594c59c6ff869ac56ef97f99aa497b8b4653a1e9"}
Jul 24 20:02:13.413465 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.413362    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:13.428963 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.428918    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" podStartSLOduration=1.428900853 podStartE2EDuration="1.428900853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:13.427633736 +0000 UTC m=+1664.522179351" watchObservedRunningTime="2026-07-24 20:02:13.428900853 +0000 UTC m=+1664.523446465"
Jul 24 20:02:13.440087 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.440062    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:13.442773 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:13.442649    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74eca896_447e_4f74_b24f_00b57d5517ec.slice/crio-8018500f5849296c7f552941c5c326d1169df2ee86007a6bbe9ad1d09d852871 WatchSource:0}: Error finding container 8018500f5849296c7f552941c5c326d1169df2ee86007a6bbe9ad1d09d852871: Status 404 returned error can't find the container with id 8018500f5849296c7f552941c5c326d1169df2ee86007a6bbe9ad1d09d852871
Jul 24 20:02:13.445528 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:13.445479    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" podStartSLOduration=0.911406809 podStartE2EDuration="1.445465115s" totalImagesPullingTime="534.058306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:13.442813569 +0000 UTC m=+1664.537359254" watchObservedRunningTime="2026-07-24 20:02:13.445465115 +0000 UTC m=+1664.540010728"
Jul 24 20:02:14.419687 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:14.419649    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" event={"ID":"74eca896-447e-4f74-b24f-00b57d5517ec","Type":"ContainerStarted","Data":"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"}
Jul 24 20:02:14.420094 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:14.419707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" event={"ID":"74eca896-447e-4f74-b24f-00b57d5517ec","Type":"ContainerStarted","Data":"8018500f5849296c7f552941c5c326d1169df2ee86007a6bbe9ad1d09d852871"}
Jul 24 20:02:14.420094 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:14.420038    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:14.435978 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:14.435930    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" podStartSLOduration=2.435917929 podStartE2EDuration="2.435917929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:14.433873676 +0000 UTC m=+1665.528419286" watchObservedRunningTime="2026-07-24 20:02:14.435917929 +0000 UTC m=+1665.530463556"
Jul 24 20:02:19.431085 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:19.431059    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:24.421763 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:24.421727    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:25.431333 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:25.431305    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:47.581921 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.581882    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:47.582385 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.582102    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" podUID="444c6353-04de-477f-9288-9435c39a16ea" containerName="maas-api" containerID="cri-o://a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5" gracePeriod=30
Jul 24 20:02:47.643360 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.643314    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:47.643578 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.643549    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" podUID="fca22a32-f95b-4537-b497-6cfa08dc3634" containerName="payload-processing" containerID="cri-o://b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e" gracePeriod=30
Jul 24 20:02:47.650384 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.650355    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:47.650608 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.650582    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" podUID="74eca896-447e-4f74-b24f-00b57d5517ec" containerName="payload-pre-processing" containerID="cri-o://9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823" gracePeriod=30
Jul 24 20:02:47.903593 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.903566    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:47.914320 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.914250    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:47.957873 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:47.957842    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:48.066539 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066499    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca\") pod \"444c6353-04de-477f-9288-9435c39a16ea\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") "
Jul 24 20:02:48.066734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066570    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls\") pod \"444c6353-04de-477f-9288-9435c39a16ea\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") "
Jul 24 20:02:48.066734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066598    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft6g2\" (UniqueName: \"kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2\") pod \"444c6353-04de-477f-9288-9435c39a16ea\" (UID: \"444c6353-04de-477f-9288-9435c39a16ea\") "
Jul 24 20:02:48.066734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066615    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume\") pod \"74eca896-447e-4f74-b24f-00b57d5517ec\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") "
Jul 24 20:02:48.066734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066669    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtv77\" (UniqueName: \"kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77\") pod \"74eca896-447e-4f74-b24f-00b57d5517ec\" (UID: \"74eca896-447e-4f74-b24f-00b57d5517ec\") "
Jul 24 20:02:48.066734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066710    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume\") pod \"fca22a32-f95b-4537-b497-6cfa08dc3634\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") "
Jul 24 20:02:48.066998 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066748    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4kwj\" (UniqueName: \"kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj\") pod \"fca22a32-f95b-4537-b497-6cfa08dc3634\" (UID: \"fca22a32-f95b-4537-b497-6cfa08dc3634\") "
Jul 24 20:02:48.066998 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.066966    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "444c6353-04de-477f-9288-9435c39a16ea" (UID: "444c6353-04de-477f-9288-9435c39a16ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:48.067110 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.067004    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74eca896-447e-4f74-b24f-00b57d5517ec" (UID: "74eca896-447e-4f74-b24f-00b57d5517ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:48.067482 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.067456    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fca22a32-f95b-4537-b497-6cfa08dc3634" (UID: "fca22a32-f95b-4537-b497-6cfa08dc3634"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:02:48.068865 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.068842    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2" (OuterVolumeSpecName: "kube-api-access-ft6g2") pod "444c6353-04de-477f-9288-9435c39a16ea" (UID: "444c6353-04de-477f-9288-9435c39a16ea"). InnerVolumeSpecName "kube-api-access-ft6g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:48.068974 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.068861    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "444c6353-04de-477f-9288-9435c39a16ea" (UID: "444c6353-04de-477f-9288-9435c39a16ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:02:48.069320 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.069291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77" (OuterVolumeSpecName: "kube-api-access-vtv77") pod "74eca896-447e-4f74-b24f-00b57d5517ec" (UID: "74eca896-447e-4f74-b24f-00b57d5517ec"). InnerVolumeSpecName "kube-api-access-vtv77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:48.069416 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.069318    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj" (OuterVolumeSpecName: "kube-api-access-b4kwj") pod "fca22a32-f95b-4537-b497-6cfa08dc3634" (UID: "fca22a32-f95b-4537-b497-6cfa08dc3634"). InnerVolumeSpecName "kube-api-access-b4kwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168065    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4kwj\" (UniqueName: \"kubernetes.io/projected/fca22a32-f95b-4537-b497-6cfa08dc3634-kube-api-access-b4kwj\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168096    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c6353-04de-477f-9288-9435c39a16ea-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168107    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c6353-04de-477f-9288-9435c39a16ea-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168116    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft6g2\" (UniqueName: \"kubernetes.io/projected/444c6353-04de-477f-9288-9435c39a16ea-kube-api-access-ft6g2\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168125    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74eca896-447e-4f74-b24f-00b57d5517ec-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168133    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtv77\" (UniqueName: \"kubernetes.io/projected/74eca896-447e-4f74-b24f-00b57d5517ec-kube-api-access-vtv77\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.168151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.168142    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fca22a32-f95b-4537-b497-6cfa08dc3634-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:02:48.605981 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.605947    2568 generic.go:356] "Generic (PLEG): container finished" podID="444c6353-04de-477f-9288-9435c39a16ea" containerID="a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5" exitCode=0
Jul 24 20:02:48.606432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.606004    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"
Jul 24 20:02:48.606432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.606042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" event={"ID":"444c6353-04de-477f-9288-9435c39a16ea","Type":"ContainerDied","Data":"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"}
Jul 24 20:02:48.606432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.606098    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps" event={"ID":"444c6353-04de-477f-9288-9435c39a16ea","Type":"ContainerDied","Data":"7e4439d7a747ae821e94b9bb594c59c6ff869ac56ef97f99aa497b8b4653a1e9"}
Jul 24 20:02:48.606432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.606126    2568 scope.go:117] "RemoveContainer" containerID="a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"
Jul 24 20:02:48.607644 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.607621    2568 generic.go:356] "Generic (PLEG): container finished" podID="74eca896-447e-4f74-b24f-00b57d5517ec" containerID="9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823" exitCode=0
Jul 24 20:02:48.607748 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.607669    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"
Jul 24 20:02:48.607802 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.607780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" event={"ID":"74eca896-447e-4f74-b24f-00b57d5517ec","Type":"ContainerDied","Data":"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"}
Jul 24 20:02:48.607859 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.607833    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz" event={"ID":"74eca896-447e-4f74-b24f-00b57d5517ec","Type":"ContainerDied","Data":"8018500f5849296c7f552941c5c326d1169df2ee86007a6bbe9ad1d09d852871"}
Jul 24 20:02:48.609311 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.609214    2568 generic.go:356] "Generic (PLEG): container finished" podID="fca22a32-f95b-4537-b497-6cfa08dc3634" containerID="b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e" exitCode=0
Jul 24 20:02:48.609311 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.609277    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" event={"ID":"fca22a32-f95b-4537-b497-6cfa08dc3634","Type":"ContainerDied","Data":"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"}
Jul 24 20:02:48.609503 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.609317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm" event={"ID":"fca22a32-f95b-4537-b497-6cfa08dc3634","Type":"ContainerDied","Data":"4cc993931b082f6edbd1459177201e7edf169b8c42c4d0a616a534be4a29dd4a"}
Jul 24 20:02:48.609503 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.609336    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"
Jul 24 20:02:48.624948 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.624928    2568 scope.go:117] "RemoveContainer" containerID="a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"
Jul 24 20:02:48.625325 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:48.625288    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5\": container with ID starting with a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5 not found: ID does not exist" containerID="a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"
Jul 24 20:02:48.625420 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.625322    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5"} err="failed to get container status \"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5\": rpc error: code = NotFound desc = could not find container \"a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5\": container with ID starting with a3de904ede41d710eb27c707f41c7b8098458c2448abca5937d6223cc7f0f7f5 not found: ID does not exist"
Jul 24 20:02:48.625420 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.625347    2568 scope.go:117] "RemoveContainer" containerID="9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"
Jul 24 20:02:48.633337 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.633316    2568 scope.go:117] "RemoveContainer" containerID="9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"
Jul 24 20:02:48.633617 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:48.633599    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823\": container with ID starting with 9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823 not found: ID does not exist" containerID="9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"
Jul 24 20:02:48.633671 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.633622    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823"} err="failed to get container status \"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823\": rpc error: code = NotFound desc = could not find container \"9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823\": container with ID starting with 9069b4b7ac71661f2e5c9885f296582408cf0fe614ab36c620d89f937018e823 not found: ID does not exist"
Jul 24 20:02:48.633671 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.633640    2568 scope.go:117] "RemoveContainer" containerID="b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"
Jul 24 20:02:48.640716 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.640695    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:48.641583 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.641569    2568 scope.go:117] "RemoveContainer" containerID="b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"
Jul 24 20:02:48.641820 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:48.641799    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e\": container with ID starting with b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e not found: ID does not exist" containerID="b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"
Jul 24 20:02:48.641875 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.641826    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e"} err="failed to get container status \"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e\": rpc error: code = NotFound desc = could not find container \"b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e\": container with ID starting with b1d30019cf1fb453952eaf95195196c2a2897d46dcdc7778b99aba901675b55e not found: ID does not exist"
Jul 24 20:02:48.644634 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.644612    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bc219aa6-f86c9d5b9-mfsvm"]
Jul 24 20:02:48.656023 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.656002    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:48.659174 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.659155    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bc219aa6-746888b9f-wlkps"]
Jul 24 20:02:48.670818 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.670797    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:48.676359 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:48.676335    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bc219aa6-8645c44f96-49djz"]
Jul 24 20:02:49.431969 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:49.431926    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="444c6353-04de-477f-9288-9435c39a16ea" path="/var/lib/kubelet/pods/444c6353-04de-477f-9288-9435c39a16ea/volumes"
Jul 24 20:02:49.432405 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:49.432384    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74eca896-447e-4f74-b24f-00b57d5517ec" path="/var/lib/kubelet/pods/74eca896-447e-4f74-b24f-00b57d5517ec/volumes"
Jul 24 20:02:49.432768 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:49.432751    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca22a32-f95b-4537-b497-6cfa08dc3634" path="/var/lib/kubelet/pods/fca22a32-f95b-4537-b497-6cfa08dc3634/volumes"
Jul 24 20:02:55.251179 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251091    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:02:55.251688 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251672    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444c6353-04de-477f-9288-9435c39a16ea" containerName="maas-api"
Jul 24 20:02:55.251743 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251691    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="444c6353-04de-477f-9288-9435c39a16ea" containerName="maas-api"
Jul 24 20:02:55.251743 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251722    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74eca896-447e-4f74-b24f-00b57d5517ec" containerName="payload-pre-processing"
Jul 24 20:02:55.251743 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251730    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="74eca896-447e-4f74-b24f-00b57d5517ec" containerName="payload-pre-processing"
Jul 24 20:02:55.252165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251745    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca22a32-f95b-4537-b497-6cfa08dc3634" containerName="payload-processing"
Jul 24 20:02:55.252165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251755    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca22a32-f95b-4537-b497-6cfa08dc3634" containerName="payload-processing"
Jul 24 20:02:55.252165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251828    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="74eca896-447e-4f74-b24f-00b57d5517ec" containerName="payload-pre-processing"
Jul 24 20:02:55.252165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251840    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="444c6353-04de-477f-9288-9435c39a16ea" containerName="maas-api"
Jul 24 20:02:55.252165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.251853    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca22a32-f95b-4537-b497-6cfa08dc3634" containerName="payload-processing"
Jul 24 20:02:55.256651 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.256628    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.259507 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.259482    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d2d8d399-serving-cert\""
Jul 24 20:02:55.267775 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.267671    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:02:55.327022 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.326993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.327234 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.327026    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szg8w\" (UniqueName: \"kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.327234 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.327121    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.428126 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.428094    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.428126 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.428135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szg8w\" (UniqueName: \"kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.428427 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.428402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.428598 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:55.428580    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-serving-cert: secret "maas-api-e2e-ait-d2d8d399-serving-cert" not found
Jul 24 20:02:55.428683 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:02:55.428667    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls podName:d231b9a2-d21f-4edb-99eb-4ad83472a49e nodeName:}" failed. No retries permitted until 2026-07-24 20:02:55.928645773 +0000 UTC m=+1707.023191381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls") pod "maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" (UID: "d231b9a2-d21f-4edb-99eb-4ad83472a49e") : secret "maas-api-e2e-ait-d2d8d399-serving-cert" not found
Jul 24 20:02:55.428892 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.428865    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.437751 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.437728    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szg8w\" (UniqueName: \"kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.932749 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.932714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:55.935177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:55.935156    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") pod \"maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:56.099612 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.099571    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:02:56.103450 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.103428    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.106304 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.106281    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d2d8d399\""
Jul 24 20:02:56.113327 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.113302    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:02:56.133858 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.133825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.134003 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.133872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n5vn\" (UniqueName: \"kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.170448 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.170413    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:56.184965 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.184906    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:02:56.190078 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.190057    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.194168 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.194146    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d2d8d399-dockercfg-mzsv8\""
Jul 24 20:02:56.214136 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.214095    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:02:56.235452 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.235390    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgkgd\" (UniqueName: \"kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.235612 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.235507    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.235612 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.235591    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.235743 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.235647    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n5vn\" (UniqueName: \"kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.236701 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.236626    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.247504 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.246558    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n5vn\" (UniqueName: \"kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn\") pod \"payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") " pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.322309 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.322281    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:02:56.325186 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:56.325156    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd231b9a2_d21f_4edb_99eb_4ad83472a49e.slice/crio-04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6 WatchSource:0}: Error finding container 04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6: Status 404 returned error can't find the container with id 04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6
Jul 24 20:02:56.336094 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.336065    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgkgd\" (UniqueName: \"kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.336193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.336136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.336852 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.336833    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.344399 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.344380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgkgd\" (UniqueName: \"kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd\") pod \"payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.415622 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.415588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:56.502694 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.502635    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:56.565979 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.565879    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:02:56.569211 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:56.569180    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81af55e_d3a8_4ff3_94f6_dfb7b3175054.slice/crio-830f9f37396b2eeb849c2af14e1cd39239401d5f85fbe22cd62602c437c5d399 WatchSource:0}: Error finding container 830f9f37396b2eeb849c2af14e1cd39239401d5f85fbe22cd62602c437c5d399: Status 404 returned error can't find the container with id 830f9f37396b2eeb849c2af14e1cd39239401d5f85fbe22cd62602c437c5d399
Jul 24 20:02:56.657906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.657838    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" event={"ID":"a81af55e-d3a8-4ff3-94f6-dfb7b3175054","Type":"ContainerStarted","Data":"830f9f37396b2eeb849c2af14e1cd39239401d5f85fbe22cd62602c437c5d399"}
Jul 24 20:02:56.659399 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.659370    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" event={"ID":"d231b9a2-d21f-4edb-99eb-4ad83472a49e","Type":"ContainerStarted","Data":"04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6"}
Jul 24 20:02:56.666675 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:56.666651    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:02:56.669663 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:02:56.669637    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6026db3d_292d_4659_bbbc_f5efca3af353.slice/crio-06eb14cd46ecd1d0e5d0b36c39e8336b47c53374053c70c840f40ff3f20701e2 WatchSource:0}: Error finding container 06eb14cd46ecd1d0e5d0b36c39e8336b47c53374053c70c840f40ff3f20701e2: Status 404 returned error can't find the container with id 06eb14cd46ecd1d0e5d0b36c39e8336b47c53374053c70c840f40ff3f20701e2
Jul 24 20:02:57.668553 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.668517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" event={"ID":"d231b9a2-d21f-4edb-99eb-4ad83472a49e","Type":"ContainerStarted","Data":"46c30050c2a8529f7201fb6b85fafccfe52ce83cd11b451248b4b76e20f37ffe"}
Jul 24 20:02:57.669019 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.668650    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:02:57.670671 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.670638    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" event={"ID":"6026db3d-292d-4659-bbbc-f5efca3af353","Type":"ContainerStarted","Data":"293dc0fb74008c7fe59f7e8deaebd58b7e1345c2e08e12ade614c776d9a08aed"}
Jul 24 20:02:57.670821 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.670703    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" event={"ID":"6026db3d-292d-4659-bbbc-f5efca3af353","Type":"ContainerStarted","Data":"06eb14cd46ecd1d0e5d0b36c39e8336b47c53374053c70c840f40ff3f20701e2"}
Jul 24 20:02:57.670821 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.670752    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:02:57.672417 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.672386    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" event={"ID":"a81af55e-d3a8-4ff3-94f6-dfb7b3175054","Type":"ContainerStarted","Data":"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"}
Jul 24 20:02:57.672527 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.672446    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:02:57.687686 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.687640    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" podStartSLOduration=2.344734951 podStartE2EDuration="2.687622395s" totalImagesPullingTime="342.887444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:57.686967038 +0000 UTC m=+1708.781512664" watchObservedRunningTime="2026-07-24 20:02:57.687622395 +0000 UTC m=+1708.782168013"
Jul 24 20:02:57.703881 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.703831    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" podStartSLOduration=1.703815771 podStartE2EDuration="1.703815771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:57.703018339 +0000 UTC m=+1708.797563957" watchObservedRunningTime="2026-07-24 20:02:57.703815771 +0000 UTC m=+1708.798361388"
Jul 24 20:02:57.721097 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:02:57.721050    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" podStartSLOduration=1.7210369170000002 podStartE2EDuration="1.721036917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:02:57.719945698 +0000 UTC m=+1708.814491312" watchObservedRunningTime="2026-07-24 20:02:57.721036917 +0000 UTC m=+1708.815582546"
Jul 24 20:03:03.685310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:03.685277    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:03:08.681866 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:08.681831    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:03:08.682504 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:08.682472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:03:31.486518 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.486482    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:03:31.487048 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.486774    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" podUID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" containerName="maas-api" containerID="cri-o://46c30050c2a8529f7201fb6b85fafccfe52ce83cd11b451248b4b76e20f37ffe" gracePeriod=30
Jul 24 20:03:31.543136 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.543101    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:03:31.543334 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.543310    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" podUID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" containerName="payload-processing" containerID="cri-o://c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423" gracePeriod=30
Jul 24 20:03:31.549133 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.549102    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:03:31.549364 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.549341    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" podUID="6026db3d-292d-4659-bbbc-f5efca3af353" containerName="payload-pre-processing" containerID="cri-o://293dc0fb74008c7fe59f7e8deaebd58b7e1345c2e08e12ade614c776d9a08aed" gracePeriod=30
Jul 24 20:03:31.844154 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.844128    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:03:31.846150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.846122    2568 generic.go:356] "Generic (PLEG): container finished" podID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" containerID="46c30050c2a8529f7201fb6b85fafccfe52ce83cd11b451248b4b76e20f37ffe" exitCode=0
Jul 24 20:03:31.846304 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.846208    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" event={"ID":"d231b9a2-d21f-4edb-99eb-4ad83472a49e","Type":"ContainerDied","Data":"46c30050c2a8529f7201fb6b85fafccfe52ce83cd11b451248b4b76e20f37ffe"}
Jul 24 20:03:31.846376 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.846320    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb" event={"ID":"d231b9a2-d21f-4edb-99eb-4ad83472a49e","Type":"ContainerDied","Data":"04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6"}
Jul 24 20:03:31.846376 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.846342    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04fdfd92f645335cd7131ae4f9808aada0727d245b40067886427b27719091e6"
Jul 24 20:03:31.847827 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.847807    2568 generic.go:356] "Generic (PLEG): container finished" podID="6026db3d-292d-4659-bbbc-f5efca3af353" containerID="293dc0fb74008c7fe59f7e8deaebd58b7e1345c2e08e12ade614c776d9a08aed" exitCode=0
Jul 24 20:03:31.847940 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.847881    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" event={"ID":"6026db3d-292d-4659-bbbc-f5efca3af353","Type":"ContainerDied","Data":"293dc0fb74008c7fe59f7e8deaebd58b7e1345c2e08e12ade614c776d9a08aed"}
Jul 24 20:03:31.848056 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.848035    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:03:31.849226 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.849208    2568 generic.go:356] "Generic (PLEG): container finished" podID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" containerID="c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423" exitCode=0
Jul 24 20:03:31.849357 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.849250    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"
Jul 24 20:03:31.849357 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.849290    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" event={"ID":"a81af55e-d3a8-4ff3-94f6-dfb7b3175054","Type":"ContainerDied","Data":"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"}
Jul 24 20:03:31.849357 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.849349    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9" event={"ID":"a81af55e-d3a8-4ff3-94f6-dfb7b3175054","Type":"ContainerDied","Data":"830f9f37396b2eeb849c2af14e1cd39239401d5f85fbe22cd62602c437c5d399"}
Jul 24 20:03:31.849517 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.849374    2568 scope.go:117] "RemoveContainer" containerID="c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"
Jul 24 20:03:31.859821 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.859798    2568 scope.go:117] "RemoveContainer" containerID="c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"
Jul 24 20:03:31.860142 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:03:31.860118    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423\": container with ID starting with c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423 not found: ID does not exist" containerID="c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"
Jul 24 20:03:31.860210 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.860154    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423"} err="failed to get container status \"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423\": rpc error: code = NotFound desc = could not find container \"c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423\": container with ID starting with c3a6a06d4ff82961e14682943f09bad6993c6fe2bdc2c9cf1744a70d5299c423 not found: ID does not exist"
Jul 24 20:03:31.870887 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.870864    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:03:31.945387 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945357    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n5vn\" (UniqueName: \"kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn\") pod \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945419    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgkgd\" (UniqueName: \"kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd\") pod \"6026db3d-292d-4659-bbbc-f5efca3af353\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945451    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") pod \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945478    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume\") pod \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\" (UID: \"a81af55e-d3a8-4ff3-94f6-dfb7b3175054\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945503    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume\") pod \"6026db3d-292d-4659-bbbc-f5efca3af353\" (UID: \"6026db3d-292d-4659-bbbc-f5efca3af353\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945549    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca\") pod \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") "
Jul 24 20:03:31.945579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945576    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szg8w\" (UniqueName: \"kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w\") pod \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\" (UID: \"d231b9a2-d21f-4edb-99eb-4ad83472a49e\") "
Jul 24 20:03:31.945928 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945902    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a81af55e-d3a8-4ff3-94f6-dfb7b3175054" (UID: "a81af55e-d3a8-4ff3-94f6-dfb7b3175054"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:03:31.945993 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945923    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6026db3d-292d-4659-bbbc-f5efca3af353" (UID: "6026db3d-292d-4659-bbbc-f5efca3af353"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:03:31.945993 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.945953    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d231b9a2-d21f-4edb-99eb-4ad83472a49e" (UID: "d231b9a2-d21f-4edb-99eb-4ad83472a49e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:03:31.947720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.947691    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d231b9a2-d21f-4edb-99eb-4ad83472a49e" (UID: "d231b9a2-d21f-4edb-99eb-4ad83472a49e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:03:31.947720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.947686    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd" (OuterVolumeSpecName: "kube-api-access-zgkgd") pod "6026db3d-292d-4659-bbbc-f5efca3af353" (UID: "6026db3d-292d-4659-bbbc-f5efca3af353"). InnerVolumeSpecName "kube-api-access-zgkgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:03:31.947720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.947692    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn" (OuterVolumeSpecName: "kube-api-access-9n5vn") pod "a81af55e-d3a8-4ff3-94f6-dfb7b3175054" (UID: "a81af55e-d3a8-4ff3-94f6-dfb7b3175054"). InnerVolumeSpecName "kube-api-access-9n5vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:03:31.947897 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:31.947782    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w" (OuterVolumeSpecName: "kube-api-access-szg8w") pod "d231b9a2-d21f-4edb-99eb-4ad83472a49e" (UID: "d231b9a2-d21f-4edb-99eb-4ad83472a49e"). InnerVolumeSpecName "kube-api-access-szg8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:03:32.047151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047068    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgkgd\" (UniqueName: \"kubernetes.io/projected/6026db3d-292d-4659-bbbc-f5efca3af353-kube-api-access-zgkgd\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047120    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d231b9a2-d21f-4edb-99eb-4ad83472a49e-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047132    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047143    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6026db3d-292d-4659-bbbc-f5efca3af353-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047152    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d231b9a2-d21f-4edb-99eb-4ad83472a49e-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047460 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047161    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szg8w\" (UniqueName: \"kubernetes.io/projected/d231b9a2-d21f-4edb-99eb-4ad83472a49e-kube-api-access-szg8w\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.047460 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.047170    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n5vn\" (UniqueName: \"kubernetes.io/projected/a81af55e-d3a8-4ff3-94f6-dfb7b3175054-kube-api-access-9n5vn\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:03:32.174318 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.174280    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:03:32.178154 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.178126    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d2d8d399-79bc6c698b-29xf9"]
Jul 24 20:03:32.856161 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.856079    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"
Jul 24 20:03:32.856161 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.856113    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67" event={"ID":"6026db3d-292d-4659-bbbc-f5efca3af353","Type":"ContainerDied","Data":"06eb14cd46ecd1d0e5d0b36c39e8336b47c53374053c70c840f40ff3f20701e2"}
Jul 24 20:03:32.856672 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.856090    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"
Jul 24 20:03:32.856672 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.856182    2568 scope.go:117] "RemoveContainer" containerID="293dc0fb74008c7fe59f7e8deaebd58b7e1345c2e08e12ade614c776d9a08aed"
Jul 24 20:03:32.884679 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.884647    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:03:32.888536 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.888512    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d2d8d399-5758c64888-h9wgb"]
Jul 24 20:03:32.899043 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.899017    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:03:32.904199 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:32.904174    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2d8d399-848f8dcbcb-smk67"]
Jul 24 20:03:33.431118 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:33.431081    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6026db3d-292d-4659-bbbc-f5efca3af353" path="/var/lib/kubelet/pods/6026db3d-292d-4659-bbbc-f5efca3af353/volumes"
Jul 24 20:03:33.431521 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:33.431504    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" path="/var/lib/kubelet/pods/a81af55e-d3a8-4ff3-94f6-dfb7b3175054/volumes"
Jul 24 20:03:33.431864 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:33.431852    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" path="/var/lib/kubelet/pods/d231b9a2-d21f-4edb-99eb-4ad83472a49e/volumes"
Jul 24 20:03:40.627590 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.627556    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:03:40.628083 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628068    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" containerName="maas-api"
Jul 24 20:03:40.628128 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628085    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" containerName="maas-api"
Jul 24 20:03:40.628128 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628111    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6026db3d-292d-4659-bbbc-f5efca3af353" containerName="payload-pre-processing"
Jul 24 20:03:40.628128 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628120    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6026db3d-292d-4659-bbbc-f5efca3af353" containerName="payload-pre-processing"
Jul 24 20:03:40.628246 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628140    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" containerName="payload-processing"
Jul 24 20:03:40.628246 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628151    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" containerName="payload-processing"
Jul 24 20:03:40.628246 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628239    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81af55e-d3a8-4ff3-94f6-dfb7b3175054" containerName="payload-processing"
Jul 24 20:03:40.628360 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628249    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d231b9a2-d21f-4edb-99eb-4ad83472a49e" containerName="maas-api"
Jul 24 20:03:40.628360 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.628276    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6026db3d-292d-4659-bbbc-f5efca3af353" containerName="payload-pre-processing"
Jul 24 20:03:40.633055 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.633034    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.635666 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.635639    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ce45103c-serving-cert\""
Jul 24 20:03:40.640689 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.640668    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:03:40.726306 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.726237    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95lrv\" (UniqueName: \"kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.726476 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.726345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.726476 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.726370    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.827623 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.827589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95lrv\" (UniqueName: \"kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.827836 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.827634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.827836 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.827659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.828346 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.828325    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.829958 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.829940    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.837594 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.837565    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95lrv\" (UniqueName: \"kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv\") pod \"maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:40.947432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:40.947395    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:41.295699 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.295672    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:03:41.298114 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:03:41.298080    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fd2d1b_07b5_440d_87ff_8619231067f9.slice/crio-bc24c7e509ace51712819b70c6552af4f3383b6e95f67466a2516d93c5244d5c WatchSource:0}: Error finding container bc24c7e509ace51712819b70c6552af4f3383b6e95f67466a2516d93c5244d5c: Status 404 returned error can't find the container with id bc24c7e509ace51712819b70c6552af4f3383b6e95f67466a2516d93c5244d5c
Jul 24 20:03:41.449154 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.449120    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:03:41.454060 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.454031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.457555 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.457527    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ce45103c-dockercfg-2tp5j\""
Jul 24 20:03:41.457821 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.457794    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ce45103c\""
Jul 24 20:03:41.465450 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.465427    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:03:41.488955 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.488926    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:03:41.492342 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.492325    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.500584 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.500558    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:03:41.533508 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.533482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.533663 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.533513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft2km\" (UniqueName: \"kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.635150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.635054    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.635839 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.635589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft2km\" (UniqueName: \"kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.635839 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.635662    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddcwx\" (UniqueName: \"kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.635957 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.635844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.636012 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.635986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.645329 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.645305    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft2km\" (UniqueName: \"kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km\") pod \"payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") " pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.737051 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.737024    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.737164 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.737080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddcwx\" (UniqueName: \"kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.737867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.737843    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.745368 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.745346    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddcwx\" (UniqueName: \"kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx\") pod \"payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.766920 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.766889    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:41.803332 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.803189    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:41.916186 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.916150    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" event={"ID":"42fd2d1b-07b5-440d-87ff-8619231067f9","Type":"ContainerStarted","Data":"1ce9e2849606908ddfe9b5f14eaed9806de0a3c360dcfc56e0633d071b63bde1"}
Jul 24 20:03:41.916358 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.916209    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" event={"ID":"42fd2d1b-07b5-440d-87ff-8619231067f9","Type":"ContainerStarted","Data":"bc24c7e509ace51712819b70c6552af4f3383b6e95f67466a2516d93c5244d5c"}
Jul 24 20:03:41.916444 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.916378    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:41.923135 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.923111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:03:41.926322 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:03:41.926253    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0eaf08a_19f0_4f61_a121_1ffd176e7a1c.slice/crio-a048ab1ad2a0630c5513d4118ef31ffac42ae9691a97ad0c59f0f2e461d8b8b2 WatchSource:0}: Error finding container a048ab1ad2a0630c5513d4118ef31ffac42ae9691a97ad0c59f0f2e461d8b8b2: Status 404 returned error can't find the container with id a048ab1ad2a0630c5513d4118ef31ffac42ae9691a97ad0c59f0f2e461d8b8b2
Jul 24 20:03:41.934845 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.934790    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" podStartSLOduration=1.5476566649999999 podStartE2EDuration="1.934769066s" totalImagesPullingTime="387.112401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:03:41.93230067 +0000 UTC m=+1753.026846284" watchObservedRunningTime="2026-07-24 20:03:41.934769066 +0000 UTC m=+1753.029314683"
Jul 24 20:03:41.962658 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:41.962634    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:03:41.967705 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:03:41.967680    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f57e19_c635_4d0d_9e77_3f6bae06c079.slice/crio-86e54bec8f5ce44e4346c051eab6c22bffed529f3bd77c18199df7de4094019d WatchSource:0}: Error finding container 86e54bec8f5ce44e4346c051eab6c22bffed529f3bd77c18199df7de4094019d: Status 404 returned error can't find the container with id 86e54bec8f5ce44e4346c051eab6c22bffed529f3bd77c18199df7de4094019d
Jul 24 20:03:42.922314 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.922275    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" event={"ID":"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c","Type":"ContainerStarted","Data":"defb466b92242192d617dfdcdec2e87efd82f12ec24ddd0588f9a8f62aed30e1"}
Jul 24 20:03:42.922755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.922342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" event={"ID":"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c","Type":"ContainerStarted","Data":"a048ab1ad2a0630c5513d4118ef31ffac42ae9691a97ad0c59f0f2e461d8b8b2"}
Jul 24 20:03:42.922755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.922372    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:03:42.923810 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.923785    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" event={"ID":"b5f57e19-c635-4d0d-9e77-3f6bae06c079","Type":"ContainerStarted","Data":"ae7510c0113a35a7113e5297b048e23fdc689f7f8af463c1397e573fd3f3abf5"}
Jul 24 20:03:42.923914 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.923838    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" event={"ID":"b5f57e19-c635-4d0d-9e77-3f6bae06c079","Type":"ContainerStarted","Data":"86e54bec8f5ce44e4346c051eab6c22bffed529f3bd77c18199df7de4094019d"}
Jul 24 20:03:42.923914 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.923871    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:42.939174 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.939130    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" podStartSLOduration=1.9391158229999998 podStartE2EDuration="1.939115823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:03:42.9380779 +0000 UTC m=+1754.032623514" watchObservedRunningTime="2026-07-24 20:03:42.939115823 +0000 UTC m=+1754.033661437"
Jul 24 20:03:42.956403 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:42.956346    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" podStartSLOduration=1.956327954 podStartE2EDuration="1.956327954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:03:42.953342887 +0000 UTC m=+1754.047888500" watchObservedRunningTime="2026-07-24 20:03:42.956327954 +0000 UTC m=+1754.050873569"
Jul 24 20:03:47.929356 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:47.929321    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:03:53.930408 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:53.930374    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:03:53.930821 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:03:53.930554    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:04:15.800772 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.800737    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:04:15.801367 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.801044    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" podUID="42fd2d1b-07b5-440d-87ff-8619231067f9" containerName="maas-api" containerID="cri-o://1ce9e2849606908ddfe9b5f14eaed9806de0a3c360dcfc56e0633d071b63bde1" gracePeriod=30
Jul 24 20:04:15.869494 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.869434    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:04:15.869776 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.869744    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" podUID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" containerName="payload-processing" containerID="cri-o://defb466b92242192d617dfdcdec2e87efd82f12ec24ddd0588f9a8f62aed30e1" gracePeriod=30
Jul 24 20:04:15.878106 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.878080    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:04:15.878324 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:15.878297    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" podUID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" containerName="payload-pre-processing" containerID="cri-o://ae7510c0113a35a7113e5297b048e23fdc689f7f8af463c1397e573fd3f3abf5" gracePeriod=30
Jul 24 20:04:16.094352 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.094189    2568 generic.go:356] "Generic (PLEG): container finished" podID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" containerID="defb466b92242192d617dfdcdec2e87efd82f12ec24ddd0588f9a8f62aed30e1" exitCode=0
Jul 24 20:04:16.094352 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.094295    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" event={"ID":"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c","Type":"ContainerDied","Data":"defb466b92242192d617dfdcdec2e87efd82f12ec24ddd0588f9a8f62aed30e1"}
Jul 24 20:04:16.096295 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.096133    2568 generic.go:356] "Generic (PLEG): container finished" podID="42fd2d1b-07b5-440d-87ff-8619231067f9" containerID="1ce9e2849606908ddfe9b5f14eaed9806de0a3c360dcfc56e0633d071b63bde1" exitCode=0
Jul 24 20:04:16.096295 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.096208    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" event={"ID":"42fd2d1b-07b5-440d-87ff-8619231067f9","Type":"ContainerDied","Data":"1ce9e2849606908ddfe9b5f14eaed9806de0a3c360dcfc56e0633d071b63bde1"}
Jul 24 20:04:16.097995 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.097970    2568 generic.go:356] "Generic (PLEG): container finished" podID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" containerID="ae7510c0113a35a7113e5297b048e23fdc689f7f8af463c1397e573fd3f3abf5" exitCode=0
Jul 24 20:04:16.098150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.098018    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" event={"ID":"b5f57e19-c635-4d0d-9e77-3f6bae06c079","Type":"ContainerDied","Data":"ae7510c0113a35a7113e5297b048e23fdc689f7f8af463c1397e573fd3f3abf5"}
Jul 24 20:04:16.129437 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.129409    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:04:16.144060 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.144036    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:04:16.184416 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.184388    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:04:16.241150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241112    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls\") pod \"42fd2d1b-07b5-440d-87ff-8619231067f9\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241181    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca\") pod \"42fd2d1b-07b5-440d-87ff-8619231067f9\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241215    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95lrv\" (UniqueName: \"kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv\") pod \"42fd2d1b-07b5-440d-87ff-8619231067f9\" (UID: \"42fd2d1b-07b5-440d-87ff-8619231067f9\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241286    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft2km\" (UniqueName: \"kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km\") pod \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241310    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddcwx\" (UniqueName: \"kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx\") pod \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241340    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume\") pod \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\" (UID: \"b5f57e19-c635-4d0d-9e77-3f6bae06c079\") "
Jul 24 20:04:16.241379 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241373    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume\") pod \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\" (UID: \"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c\") "
Jul 24 20:04:16.241688 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241626    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42fd2d1b-07b5-440d-87ff-8619231067f9" (UID: "42fd2d1b-07b5-440d-87ff-8619231067f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:04:16.241980 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.241955    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" (UID: "c0eaf08a-19f0-4f61-a121-1ffd176e7a1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:04:16.242069 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.242012    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5f57e19-c635-4d0d-9e77-3f6bae06c079" (UID: "b5f57e19-c635-4d0d-9e77-3f6bae06c079"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:04:16.243461 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.243423    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx" (OuterVolumeSpecName: "kube-api-access-ddcwx") pod "b5f57e19-c635-4d0d-9e77-3f6bae06c079" (UID: "b5f57e19-c635-4d0d-9e77-3f6bae06c079"). InnerVolumeSpecName "kube-api-access-ddcwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:04:16.243587 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.243512    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv" (OuterVolumeSpecName: "kube-api-access-95lrv") pod "42fd2d1b-07b5-440d-87ff-8619231067f9" (UID: "42fd2d1b-07b5-440d-87ff-8619231067f9"). InnerVolumeSpecName "kube-api-access-95lrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:04:16.243724 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.243707    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km" (OuterVolumeSpecName: "kube-api-access-ft2km") pod "c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" (UID: "c0eaf08a-19f0-4f61-a121-1ffd176e7a1c"). InnerVolumeSpecName "kube-api-access-ft2km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:04:16.243783 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.243733    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42fd2d1b-07b5-440d-87ff-8619231067f9" (UID: "42fd2d1b-07b5-440d-87ff-8619231067f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342879    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5f57e19-c635-4d0d-9e77-3f6bae06c079-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342907    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342921    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42fd2d1b-07b5-440d-87ff-8619231067f9-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342934    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2d1b-07b5-440d-87ff-8619231067f9-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342946    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95lrv\" (UniqueName: \"kubernetes.io/projected/42fd2d1b-07b5-440d-87ff-8619231067f9-kube-api-access-95lrv\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342959    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft2km\" (UniqueName: \"kubernetes.io/projected/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c-kube-api-access-ft2km\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:16.342970 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:16.342972    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddcwx\" (UniqueName: \"kubernetes.io/projected/b5f57e19-c635-4d0d-9e77-3f6bae06c079-kube-api-access-ddcwx\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:04:17.104432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.104348    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"
Jul 24 20:04:17.104432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.104367    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9" event={"ID":"c0eaf08a-19f0-4f61-a121-1ffd176e7a1c","Type":"ContainerDied","Data":"a048ab1ad2a0630c5513d4118ef31ffac42ae9691a97ad0c59f0f2e461d8b8b2"}
Jul 24 20:04:17.104935 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.104440    2568 scope.go:117] "RemoveContainer" containerID="defb466b92242192d617dfdcdec2e87efd82f12ec24ddd0588f9a8f62aed30e1"
Jul 24 20:04:17.105783 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.105744    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl" event={"ID":"42fd2d1b-07b5-440d-87ff-8619231067f9","Type":"ContainerDied","Data":"bc24c7e509ace51712819b70c6552af4f3383b6e95f67466a2516d93c5244d5c"}
Jul 24 20:04:17.105913 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.105764    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"
Jul 24 20:04:17.107280 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.107232    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5" event={"ID":"b5f57e19-c635-4d0d-9e77-3f6bae06c079","Type":"ContainerDied","Data":"86e54bec8f5ce44e4346c051eab6c22bffed529f3bd77c18199df7de4094019d"}
Jul 24 20:04:17.107444 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.107243    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"
Jul 24 20:04:17.114936 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.114912    2568 scope.go:117] "RemoveContainer" containerID="1ce9e2849606908ddfe9b5f14eaed9806de0a3c360dcfc56e0633d071b63bde1"
Jul 24 20:04:17.123314 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.123296    2568 scope.go:117] "RemoveContainer" containerID="ae7510c0113a35a7113e5297b048e23fdc689f7f8af463c1397e573fd3f3abf5"
Jul 24 20:04:17.133290 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.133253    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:04:17.137961 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.137939    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ce45103c-647fb9d798-26dk9"]
Jul 24 20:04:17.149564 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.149539    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:04:17.154067 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.154043    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ce45103c-7c5b599dfd-fbldl"]
Jul 24 20:04:17.167573 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.167547    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:04:17.171366 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.171344    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ce45103c-6c6fd87c7b-7xsj5"]
Jul 24 20:04:17.430928 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.430851    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42fd2d1b-07b5-440d-87ff-8619231067f9" path="/var/lib/kubelet/pods/42fd2d1b-07b5-440d-87ff-8619231067f9/volumes"
Jul 24 20:04:17.431225 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.431210    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" path="/var/lib/kubelet/pods/b5f57e19-c635-4d0d-9e77-3f6bae06c079/volumes"
Jul 24 20:04:17.431574 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:17.431561    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" path="/var/lib/kubelet/pods/c0eaf08a-19f0-4f61-a121-1ffd176e7a1c/volumes"
Jul 24 20:04:29.431772 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:29.431739    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:04:29.440339 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:04:29.440314    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:06:02.903886 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.903796    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904199    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" containerName="payload-pre-processing"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904215    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" containerName="payload-pre-processing"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904236    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" containerName="payload-processing"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904242    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" containerName="payload-processing"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904249    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42fd2d1b-07b5-440d-87ff-8619231067f9" containerName="maas-api"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904269    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="42fd2d1b-07b5-440d-87ff-8619231067f9" containerName="maas-api"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904342    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f57e19-c635-4d0d-9e77-3f6bae06c079" containerName="payload-pre-processing"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904355    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="42fd2d1b-07b5-440d-87ff-8619231067f9" containerName="maas-api"
Jul 24 20:06:02.904466 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.904361    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaf08a-19f0-4f61-a121-1ffd176e7a1c" containerName="payload-processing"
Jul 24 20:06:02.907719 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.907696    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:02.910383 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.910358    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c86ab55b-serving-cert\""
Jul 24 20:06:02.919758 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:02.919586    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:03.007166 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.007131    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkm5n\" (UniqueName: \"kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.007370 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.007194    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.007370 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.007318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.107877 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.107826    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.108074 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.107927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkm5n\" (UniqueName: \"kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.108074 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.108001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.108199 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:06:03.108095    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-serving-cert: secret "maas-api-e2e-mt-c86ab55b-serving-cert" not found
Jul 24 20:06:03.108199 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:06:03.108166    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls podName:c71e9edb-2e3f-4810-8adc-e9058be028db nodeName:}" failed. No retries permitted until 2026-07-24 20:06:03.608148095 +0000 UTC m=+1894.702693685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls") pod "maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" (UID: "c71e9edb-2e3f-4810-8adc-e9058be028db") : secret "maas-api-e2e-mt-c86ab55b-serving-cert" not found
Jul 24 20:06:03.108601 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.108577    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.117202 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.117179    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkm5n\" (UniqueName: \"kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.613721 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.613675    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.616080 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.616049    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") pod \"maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.699906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.699869    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:03.703473 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.703456    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.706786 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.706761    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c86ab55b\""
Jul 24 20:06:03.711911 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.711717    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:03.730944 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.730913    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:03.734547 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.734531    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.737459 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.737429    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c86ab55b-dockercfg-9fnqg\""
Jul 24 20:06:03.743052 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.743029    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:03.815452 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.815414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.815661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.815470    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.815661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.815574    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbpwc\" (UniqueName: \"kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.815832 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.815666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58w2l\" (UniqueName: \"kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.821330 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.821303    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:03.916768 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.916739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpwc\" (UniqueName: \"kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.917129 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.916807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58w2l\" (UniqueName: \"kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.917129 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.916880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.917129 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.916935    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.917909 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.917779    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.917909 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.917819    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.925009 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.924961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbpwc\" (UniqueName: \"kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc\") pod \"payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") " pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:03.925009 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.924996    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58w2l\" (UniqueName: \"kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l\") pod \"payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:03.977394 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:03.977357    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:03.979352 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:06:03.979322    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71e9edb_2e3f_4810_8adc_e9058be028db.slice/crio-82ded0f035c4637ccec286ae7d434d03fa433cae1f445f97246dfe61511f09a2 WatchSource:0}: Error finding container 82ded0f035c4637ccec286ae7d434d03fa433cae1f445f97246dfe61511f09a2: Status 404 returned error can't find the container with id 82ded0f035c4637ccec286ae7d434d03fa433cae1f445f97246dfe61511f09a2
Jul 24 20:06:04.015966 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.015934    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:04.046179 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.046150    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:04.175232 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.175199    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:04.177246 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:06:04.177207    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc115819c_8269_4eac_9fea_789506b8ad93.slice/crio-383baad81a79ffdd02b537e5cb24f0ccb7dbc3c5fe23e35bc60e41c3762e1eda WatchSource:0}: Error finding container 383baad81a79ffdd02b537e5cb24f0ccb7dbc3c5fe23e35bc60e41c3762e1eda: Status 404 returned error can't find the container with id 383baad81a79ffdd02b537e5cb24f0ccb7dbc3c5fe23e35bc60e41c3762e1eda
Jul 24 20:06:04.207451 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.207384    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:04.216430 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:06:04.216401    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20aebe13_1c90_4d06_95d7_c980f4d4121d.slice/crio-85bc2ae2a1248a8cd7c9a8643a055a1d4bfa532bc1e02a3068e7c9d1ae086fb1 WatchSource:0}: Error finding container 85bc2ae2a1248a8cd7c9a8643a055a1d4bfa532bc1e02a3068e7c9d1ae086fb1: Status 404 returned error can't find the container with id 85bc2ae2a1248a8cd7c9a8643a055a1d4bfa532bc1e02a3068e7c9d1ae086fb1
Jul 24 20:06:04.651616 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.651582    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" event={"ID":"20aebe13-1c90-4d06-95d7-c980f4d4121d","Type":"ContainerStarted","Data":"834a7ab78ae80161c6aad9b711af19f4421922e3f7bf7d2d064718b4de4f594b"}
Jul 24 20:06:04.651804 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.651630    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" event={"ID":"20aebe13-1c90-4d06-95d7-c980f4d4121d","Type":"ContainerStarted","Data":"85bc2ae2a1248a8cd7c9a8643a055a1d4bfa532bc1e02a3068e7c9d1ae086fb1"}
Jul 24 20:06:04.651804 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.651781    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:04.653530 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.653496    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" event={"ID":"c71e9edb-2e3f-4810-8adc-e9058be028db","Type":"ContainerStarted","Data":"04cf762bc4056dbb21935648aa88005f84afc1186d89577703bfe00c0a66183e"}
Jul 24 20:06:04.653671 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.653555    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" event={"ID":"c71e9edb-2e3f-4810-8adc-e9058be028db","Type":"ContainerStarted","Data":"82ded0f035c4637ccec286ae7d434d03fa433cae1f445f97246dfe61511f09a2"}
Jul 24 20:06:04.654088 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.654063    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:04.655883 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.655853    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" event={"ID":"c115819c-8269-4eac-9fea-789506b8ad93","Type":"ContainerStarted","Data":"d05d654981cf46cb29649956bf99215399de84f6815444813b079721c912daad"}
Jul 24 20:06:04.655992 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.655907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" event={"ID":"c115819c-8269-4eac-9fea-789506b8ad93","Type":"ContainerStarted","Data":"383baad81a79ffdd02b537e5cb24f0ccb7dbc3c5fe23e35bc60e41c3762e1eda"}
Jul 24 20:06:04.656400 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.656377    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:04.673052 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.672993    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" podStartSLOduration=1.672975933 podStartE2EDuration="1.672975933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:06:04.672156242 +0000 UTC m=+1895.766701869" watchObservedRunningTime="2026-07-24 20:06:04.672975933 +0000 UTC m=+1895.767521545"
Jul 24 20:06:04.706776 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.706677    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" podStartSLOduration=2.23364467 podStartE2EDuration="2.706656832s" totalImagesPullingTime="473.012162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:06:04.706099608 +0000 UTC m=+1895.800645232" watchObservedRunningTime="2026-07-24 20:06:04.706656832 +0000 UTC m=+1895.801202445"
Jul 24 20:06:04.707450 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:04.707423    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" podStartSLOduration=1.707416265 podStartE2EDuration="1.707416265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:06:04.688606928 +0000 UTC m=+1895.783152542" watchObservedRunningTime="2026-07-24 20:06:04.707416265 +0000 UTC m=+1895.801961878"
Jul 24 20:06:11.675683 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:11.675652    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:15.664902 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:15.664873    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:16.672001 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:16.671960    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" podUID="c115819c-8269-4eac-9fea-789506b8ad93" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.65:9004: connect: connection refused"
Jul 24 20:06:24.577905 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.577864    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:24.578403 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.578114    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" podUID="c71e9edb-2e3f-4810-8adc-e9058be028db" containerName="maas-api" containerID="cri-o://04cf762bc4056dbb21935648aa88005f84afc1186d89577703bfe00c0a66183e" gracePeriod=30
Jul 24 20:06:24.637582 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.637548    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:24.637813 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.637786    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" podUID="20aebe13-1c90-4d06-95d7-c980f4d4121d" containerName="payload-processing" containerID="cri-o://834a7ab78ae80161c6aad9b711af19f4421922e3f7bf7d2d064718b4de4f594b" gracePeriod=30
Jul 24 20:06:24.643053 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.642758    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:24.643053 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.642990    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" podUID="c115819c-8269-4eac-9fea-789506b8ad93" containerName="payload-pre-processing" containerID="cri-o://d05d654981cf46cb29649956bf99215399de84f6815444813b079721c912daad" gracePeriod=30
Jul 24 20:06:24.644700 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.644679    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:24.776873 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.776832    2568 generic.go:356] "Generic (PLEG): container finished" podID="20aebe13-1c90-4d06-95d7-c980f4d4121d" containerID="834a7ab78ae80161c6aad9b711af19f4421922e3f7bf7d2d064718b4de4f594b" exitCode=0
Jul 24 20:06:24.777046 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.776914    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" event={"ID":"20aebe13-1c90-4d06-95d7-c980f4d4121d","Type":"ContainerDied","Data":"834a7ab78ae80161c6aad9b711af19f4421922e3f7bf7d2d064718b4de4f594b"}
Jul 24 20:06:24.778505 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.778478    2568 generic.go:356] "Generic (PLEG): container finished" podID="c71e9edb-2e3f-4810-8adc-e9058be028db" containerID="04cf762bc4056dbb21935648aa88005f84afc1186d89577703bfe00c0a66183e" exitCode=0
Jul 24 20:06:24.778628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.778566    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" event={"ID":"c71e9edb-2e3f-4810-8adc-e9058be028db","Type":"ContainerDied","Data":"04cf762bc4056dbb21935648aa88005f84afc1186d89577703bfe00c0a66183e"}
Jul 24 20:06:24.780119 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.780096    2568 generic.go:356] "Generic (PLEG): container finished" podID="c115819c-8269-4eac-9fea-789506b8ad93" containerID="d05d654981cf46cb29649956bf99215399de84f6815444813b079721c912daad" exitCode=0
Jul 24 20:06:24.780205 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.780133    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" event={"ID":"c115819c-8269-4eac-9fea-789506b8ad93","Type":"ContainerDied","Data":"d05d654981cf46cb29649956bf99215399de84f6815444813b079721c912daad"}
Jul 24 20:06:24.917173 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.917144    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:24.949697 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.949670    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:24.959881 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:24.959855    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:25.013694 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013665    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca\") pod \"c71e9edb-2e3f-4810-8adc-e9058be028db\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") "
Jul 24 20:06:25.013867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013716    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume\") pod \"20aebe13-1c90-4d06-95d7-c980f4d4121d\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") "
Jul 24 20:06:25.013867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013739    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") pod \"c71e9edb-2e3f-4810-8adc-e9058be028db\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") "
Jul 24 20:06:25.013867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013757    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58w2l\" (UniqueName: \"kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l\") pod \"c115819c-8269-4eac-9fea-789506b8ad93\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") "
Jul 24 20:06:25.013867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013787    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume\") pod \"c115819c-8269-4eac-9fea-789506b8ad93\" (UID: \"c115819c-8269-4eac-9fea-789506b8ad93\") "
Jul 24 20:06:25.013867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013824    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkm5n\" (UniqueName: \"kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n\") pod \"c71e9edb-2e3f-4810-8adc-e9058be028db\" (UID: \"c71e9edb-2e3f-4810-8adc-e9058be028db\") "
Jul 24 20:06:25.014157 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.013882    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbpwc\" (UniqueName: \"kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc\") pod \"20aebe13-1c90-4d06-95d7-c980f4d4121d\" (UID: \"20aebe13-1c90-4d06-95d7-c980f4d4121d\") "
Jul 24 20:06:25.014157 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.014084    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c71e9edb-2e3f-4810-8adc-e9058be028db" (UID: "c71e9edb-2e3f-4810-8adc-e9058be028db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:06:25.014157 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.014130    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20aebe13-1c90-4d06-95d7-c980f4d4121d" (UID: "20aebe13-1c90-4d06-95d7-c980f4d4121d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:06:25.014565 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.014541    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c115819c-8269-4eac-9fea-789506b8ad93" (UID: "c115819c-8269-4eac-9fea-789506b8ad93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:06:25.016386 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.016357    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c71e9edb-2e3f-4810-8adc-e9058be028db" (UID: "c71e9edb-2e3f-4810-8adc-e9058be028db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:06:25.016473 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.016412    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n" (OuterVolumeSpecName: "kube-api-access-tkm5n") pod "c71e9edb-2e3f-4810-8adc-e9058be028db" (UID: "c71e9edb-2e3f-4810-8adc-e9058be028db"). InnerVolumeSpecName "kube-api-access-tkm5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:06:25.016523 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.016478    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc" (OuterVolumeSpecName: "kube-api-access-jbpwc") pod "20aebe13-1c90-4d06-95d7-c980f4d4121d" (UID: "20aebe13-1c90-4d06-95d7-c980f4d4121d"). InnerVolumeSpecName "kube-api-access-jbpwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:06:25.016523 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.016504    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l" (OuterVolumeSpecName: "kube-api-access-58w2l") pod "c115819c-8269-4eac-9fea-789506b8ad93" (UID: "c115819c-8269-4eac-9fea-789506b8ad93"). InnerVolumeSpecName "kube-api-access-58w2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115250    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbpwc\" (UniqueName: \"kubernetes.io/projected/20aebe13-1c90-4d06-95d7-c980f4d4121d-kube-api-access-jbpwc\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115348    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c71e9edb-2e3f-4810-8adc-e9058be028db-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115359    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20aebe13-1c90-4d06-95d7-c980f4d4121d-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115369    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c71e9edb-2e3f-4810-8adc-e9058be028db-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115378    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58w2l\" (UniqueName: \"kubernetes.io/projected/c115819c-8269-4eac-9fea-789506b8ad93-kube-api-access-58w2l\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115387    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c115819c-8269-4eac-9fea-789506b8ad93-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.115393 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.115396    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkm5n\" (UniqueName: \"kubernetes.io/projected/c71e9edb-2e3f-4810-8adc-e9058be028db-kube-api-access-tkm5n\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:06:25.787777 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.787689    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"
Jul 24 20:06:25.787777 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.787709    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j" event={"ID":"20aebe13-1c90-4d06-95d7-c980f4d4121d","Type":"ContainerDied","Data":"85bc2ae2a1248a8cd7c9a8643a055a1d4bfa532bc1e02a3068e7c9d1ae086fb1"}
Jul 24 20:06:25.788292 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.787782    2568 scope.go:117] "RemoveContainer" containerID="834a7ab78ae80161c6aad9b711af19f4421922e3f7bf7d2d064718b4de4f594b"
Jul 24 20:06:25.789306 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.789088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6" event={"ID":"c71e9edb-2e3f-4810-8adc-e9058be028db","Type":"ContainerDied","Data":"82ded0f035c4637ccec286ae7d434d03fa433cae1f445f97246dfe61511f09a2"}
Jul 24 20:06:25.789306 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.789107    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"
Jul 24 20:06:25.790596 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.790576    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"
Jul 24 20:06:25.790727 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.790574    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh" event={"ID":"c115819c-8269-4eac-9fea-789506b8ad93","Type":"ContainerDied","Data":"383baad81a79ffdd02b537e5cb24f0ccb7dbc3c5fe23e35bc60e41c3762e1eda"}
Jul 24 20:06:25.797670 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.797652    2568 scope.go:117] "RemoveContainer" containerID="04cf762bc4056dbb21935648aa88005f84afc1186d89577703bfe00c0a66183e"
Jul 24 20:06:25.805343 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.805325    2568 scope.go:117] "RemoveContainer" containerID="d05d654981cf46cb29649956bf99215399de84f6815444813b079721c912daad"
Jul 24 20:06:25.811169 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.811147    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:25.813850 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.813830    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c86ab55b-5778bbc4cb-z77p6"]
Jul 24 20:06:25.823655 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.823634    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:25.827067 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.827045    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c86ab55b-5cfc67ff8b-qhh5j"]
Jul 24 20:06:25.845012 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.844990    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:25.850349 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:25.850329    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c86ab55b-76f4dcbcd4-89xfh"]
Jul 24 20:06:27.430976 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:27.430942    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20aebe13-1c90-4d06-95d7-c980f4d4121d" path="/var/lib/kubelet/pods/20aebe13-1c90-4d06-95d7-c980f4d4121d/volumes"
Jul 24 20:06:27.431387 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:27.431319    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c115819c-8269-4eac-9fea-789506b8ad93" path="/var/lib/kubelet/pods/c115819c-8269-4eac-9fea-789506b8ad93/volumes"
Jul 24 20:06:27.431675 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:27.431661    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c71e9edb-2e3f-4810-8adc-e9058be028db" path="/var/lib/kubelet/pods/c71e9edb-2e3f-4810-8adc-e9058be028db/volumes"
Jul 24 20:06:58.115180 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115143    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115655    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c115819c-8269-4eac-9fea-789506b8ad93" containerName="payload-pre-processing"
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115673    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c115819c-8269-4eac-9fea-789506b8ad93" containerName="payload-pre-processing"
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115683    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20aebe13-1c90-4d06-95d7-c980f4d4121d" containerName="payload-processing"
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115690    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="20aebe13-1c90-4d06-95d7-c980f4d4121d" containerName="payload-processing"
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115708    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c71e9edb-2e3f-4810-8adc-e9058be028db" containerName="maas-api"
Jul 24 20:06:58.115759 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115713    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c71e9edb-2e3f-4810-8adc-e9058be028db" containerName="maas-api"
Jul 24 20:06:58.116080 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115773    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="20aebe13-1c90-4d06-95d7-c980f4d4121d" containerName="payload-processing"
Jul 24 20:06:58.116080 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115788    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c71e9edb-2e3f-4810-8adc-e9058be028db" containerName="maas-api"
Jul 24 20:06:58.116080 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.115799    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c115819c-8269-4eac-9fea-789506b8ad93" containerName="payload-pre-processing"
Jul 24 20:06:58.119065 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.119043    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.121930 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.121905    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-74e2adbb-serving-cert\""
Jul 24 20:06:58.131473 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.131421    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:06:58.217357 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.217313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.217567 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.217419    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlr5b\" (UniqueName: \"kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.217567 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.217448    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.318586 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.318546    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.318787 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.318615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlr5b\" (UniqueName: \"kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.318787 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.318636    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.318787 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:06:58.318710    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-serving-cert: secret "maas-api-e2e-mt-74e2adbb-serving-cert" not found
Jul 24 20:06:58.318951 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:06:58.318790    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls podName:757e5c21-8e65-4136-aa93-c29387580f7a nodeName:}" failed. No retries permitted until 2026-07-24 20:06:58.81876949 +0000 UTC m=+1949.913315099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls") pod "maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" (UID: "757e5c21-8e65-4136-aa93-c29387580f7a") : secret "maas-api-e2e-mt-74e2adbb-serving-cert" not found
Jul 24 20:06:58.319224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.319205    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.327532 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.327511    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlr5b\" (UniqueName: \"kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.823405 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.823364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.825858 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.825831    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") pod \"maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:58.888723 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.888680    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:06:58.892949 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.892903    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:58.896456 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.896378    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-74e2adbb\""
Jul 24 20:06:58.896598 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.896528    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-74e2adbb-dockercfg-bhjx2\""
Jul 24 20:06:58.904168 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.904142    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:06:58.907419 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.907399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:58.910331 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.910307    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:06:58.917837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:58.917816    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:06:59.025440 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.025391    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.025440 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.025432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbbd\" (UniqueName: \"kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.025693 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.025470    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58pm8\" (UniqueName: \"kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.025693 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.025496    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.031720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.031684    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:59.126971 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.126932    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.127391 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.126984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkbbd\" (UniqueName: \"kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.127391 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.127048    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58pm8\" (UniqueName: \"kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.127391 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.127094    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.127852 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.127825    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.128035 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.128006    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.135445 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.135405    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58pm8\" (UniqueName: \"kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8\") pod \"payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.135551 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.135521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkbbd\" (UniqueName: \"kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd\") pod \"payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") " pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.206117 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.206078    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.218408 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.218381    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.356565 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.356484    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:06:59.393654 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.393616    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:06:59.395705 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:06:59.395682    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757e5c21_8e65_4136_aa93_c29387580f7a.slice/crio-98c94ce00f72cb8a9d2590cc9ddc2081c8bbc00eb28ceec48b1d40e132a2f5e7 WatchSource:0}: Error finding container 98c94ce00f72cb8a9d2590cc9ddc2081c8bbc00eb28ceec48b1d40e132a2f5e7: Status 404 returned error can't find the container with id 98c94ce00f72cb8a9d2590cc9ddc2081c8bbc00eb28ceec48b1d40e132a2f5e7
Jul 24 20:06:59.397336 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.397317    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:06:59.594977 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.594946    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:06:59.596833 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:06:59.596807    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488ff1ae_810d_4298_b92f_32337aae0fe6.slice/crio-e2b615de9715e4f337d921640a623da5ff5769ed44dda5aaf9c510c2da2a83a5 WatchSource:0}: Error finding container e2b615de9715e4f337d921640a623da5ff5769ed44dda5aaf9c510c2da2a83a5: Status 404 returned error can't find the container with id e2b615de9715e4f337d921640a623da5ff5769ed44dda5aaf9c510c2da2a83a5
Jul 24 20:06:59.961229 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.961186    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" event={"ID":"78861b5c-a5f8-47ac-967e-57833dbd7ce4","Type":"ContainerStarted","Data":"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"}
Jul 24 20:06:59.961440 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.961286    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" event={"ID":"78861b5c-a5f8-47ac-967e-57833dbd7ce4","Type":"ContainerStarted","Data":"43a7a98c1ab5802898dc6f5674a51059fabca5012cc5e4fef0c1b37a4f218fee"}
Jul 24 20:06:59.961939 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.961916    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:06:59.963928 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.963896    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" event={"ID":"488ff1ae-810d-4298-b92f-32337aae0fe6","Type":"ContainerStarted","Data":"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"}
Jul 24 20:06:59.964052 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.963951    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" event={"ID":"488ff1ae-810d-4298-b92f-32337aae0fe6","Type":"ContainerStarted","Data":"e2b615de9715e4f337d921640a623da5ff5769ed44dda5aaf9c510c2da2a83a5"}
Jul 24 20:06:59.964052 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.963982    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:06:59.966403 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.966379    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" event={"ID":"757e5c21-8e65-4136-aa93-c29387580f7a","Type":"ContainerStarted","Data":"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"}
Jul 24 20:06:59.966505 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.966424    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" event={"ID":"757e5c21-8e65-4136-aa93-c29387580f7a","Type":"ContainerStarted","Data":"98c94ce00f72cb8a9d2590cc9ddc2081c8bbc00eb28ceec48b1d40e132a2f5e7"}
Jul 24 20:06:59.966843 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.966823    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:06:59.977289 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.977223    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" podStartSLOduration=1.977205849 podStartE2EDuration="1.977205849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:06:59.976894496 +0000 UTC m=+1951.071440122" watchObservedRunningTime="2026-07-24 20:06:59.977205849 +0000 UTC m=+1951.071751454"
Jul 24 20:06:59.994664 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:06:59.994617    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" podStartSLOduration=1.9946008050000001 podStartE2EDuration="1.994600805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:06:59.992751181 +0000 UTC m=+1951.087296796" watchObservedRunningTime="2026-07-24 20:06:59.994600805 +0000 UTC m=+1951.089146419"
Jul 24 20:07:00.010906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:07:00.010862    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" podStartSLOduration=1.622663174 podStartE2EDuration="2.010843656s" totalImagesPullingTime="388.180482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:06:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:07:00.009555188 +0000 UTC m=+1951.104100803" watchObservedRunningTime="2026-07-24 20:07:00.010843656 +0000 UTC m=+1951.105389268"
Jul 24 20:07:05.977573 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:07:05.977539    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:07:10.974287 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:07:10.974228    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:07:11.978734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:07:11.978685    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.68:9004: connect: connection refused"
Jul 24 20:07:21.977354 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:07:21.977238    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:08:12.762197 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.762158    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:08:12.762815 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.762408    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" podUID="757e5c21-8e65-4136-aa93-c29387580f7a" containerName="maas-api" containerID="cri-o://eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b" gracePeriod=30
Jul 24 20:08:12.818403 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.818368    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:08:12.818621 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.818576    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" podUID="488ff1ae-810d-4298-b92f-32337aae0fe6" containerName="payload-processing" containerID="cri-o://667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d" gracePeriod=30
Jul 24 20:08:12.827657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.827630    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:08:12.827872 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:12.827846    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerName="payload-pre-processing" containerID="cri-o://47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4" gracePeriod=30
Jul 24 20:08:13.084952 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.084915    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:08:13.127515 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.127496    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:08:13.134576 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.134554    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165626    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlr5b\" (UniqueName: \"kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b\") pod \"757e5c21-8e65-4136-aa93-c29387580f7a\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165671    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58pm8\" (UniqueName: \"kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8\") pod \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165713    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") pod \"757e5c21-8e65-4136-aa93-c29387580f7a\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165773    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkbbd\" (UniqueName: \"kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd\") pod \"488ff1ae-810d-4298-b92f-32337aae0fe6\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165809    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca\") pod \"757e5c21-8e65-4136-aa93-c29387580f7a\" (UID: \"757e5c21-8e65-4136-aa93-c29387580f7a\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165831    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume\") pod \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\" (UID: \"78861b5c-a5f8-47ac-967e-57833dbd7ce4\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.165863    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume\") pod \"488ff1ae-810d-4298-b92f-32337aae0fe6\" (UID: \"488ff1ae-810d-4298-b92f-32337aae0fe6\") "
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.166636    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "488ff1ae-810d-4298-b92f-32337aae0fe6" (UID: "488ff1ae-810d-4298-b92f-32337aae0fe6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:08:13.166698 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.166641    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78861b5c-a5f8-47ac-967e-57833dbd7ce4" (UID: "78861b5c-a5f8-47ac-967e-57833dbd7ce4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:08:13.167309 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.166792    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "757e5c21-8e65-4136-aa93-c29387580f7a" (UID: "757e5c21-8e65-4136-aa93-c29387580f7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:08:13.168248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.168217    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8" (OuterVolumeSpecName: "kube-api-access-58pm8") pod "78861b5c-a5f8-47ac-967e-57833dbd7ce4" (UID: "78861b5c-a5f8-47ac-967e-57833dbd7ce4"). InnerVolumeSpecName "kube-api-access-58pm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:08:13.168425 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.168405    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b" (OuterVolumeSpecName: "kube-api-access-qlr5b") pod "757e5c21-8e65-4136-aa93-c29387580f7a" (UID: "757e5c21-8e65-4136-aa93-c29387580f7a"). InnerVolumeSpecName "kube-api-access-qlr5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:08:13.169026 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.169001    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "757e5c21-8e65-4136-aa93-c29387580f7a" (UID: "757e5c21-8e65-4136-aa93-c29387580f7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:08:13.169200 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.169180    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd" (OuterVolumeSpecName: "kube-api-access-xkbbd") pod "488ff1ae-810d-4298-b92f-32337aae0fe6" (UID: "488ff1ae-810d-4298-b92f-32337aae0fe6"). InnerVolumeSpecName "kube-api-access-xkbbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:08:13.266690 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266650    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/757e5c21-8e65-4136-aa93-c29387580f7a-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266690 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266690    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkbbd\" (UniqueName: \"kubernetes.io/projected/488ff1ae-810d-4298-b92f-32337aae0fe6-kube-api-access-xkbbd\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266706    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/757e5c21-8e65-4136-aa93-c29387580f7a-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266721    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78861b5c-a5f8-47ac-967e-57833dbd7ce4-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266736    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/488ff1ae-810d-4298-b92f-32337aae0fe6-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266749    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlr5b\" (UniqueName: \"kubernetes.io/projected/757e5c21-8e65-4136-aa93-c29387580f7a-kube-api-access-qlr5b\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.266880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.266763    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58pm8\" (UniqueName: \"kubernetes.io/projected/78861b5c-a5f8-47ac-967e-57833dbd7ce4-kube-api-access-58pm8\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:08:13.329453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.329358    2568 generic.go:356] "Generic (PLEG): container finished" podID="488ff1ae-810d-4298-b92f-32337aae0fe6" containerID="667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d" exitCode=0
Jul 24 20:08:13.329453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.329410    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"
Jul 24 20:08:13.329656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.329441    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" event={"ID":"488ff1ae-810d-4298-b92f-32337aae0fe6","Type":"ContainerDied","Data":"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"}
Jul 24 20:08:13.329656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.329507    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l" event={"ID":"488ff1ae-810d-4298-b92f-32337aae0fe6","Type":"ContainerDied","Data":"e2b615de9715e4f337d921640a623da5ff5769ed44dda5aaf9c510c2da2a83a5"}
Jul 24 20:08:13.329656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.329536    2568 scope.go:117] "RemoveContainer" containerID="667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"
Jul 24 20:08:13.330833 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.330800    2568 generic.go:356] "Generic (PLEG): container finished" podID="757e5c21-8e65-4136-aa93-c29387580f7a" containerID="eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b" exitCode=0
Jul 24 20:08:13.330897 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.330840    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"
Jul 24 20:08:13.330897 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.330857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" event={"ID":"757e5c21-8e65-4136-aa93-c29387580f7a","Type":"ContainerDied","Data":"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"}
Jul 24 20:08:13.331001 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.330919    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9" event={"ID":"757e5c21-8e65-4136-aa93-c29387580f7a","Type":"ContainerDied","Data":"98c94ce00f72cb8a9d2590cc9ddc2081c8bbc00eb28ceec48b1d40e132a2f5e7"}
Jul 24 20:08:13.332394 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.332372    2568 generic.go:356] "Generic (PLEG): container finished" podID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerID="47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4" exitCode=0
Jul 24 20:08:13.332516 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.332423    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" event={"ID":"78861b5c-a5f8-47ac-967e-57833dbd7ce4","Type":"ContainerDied","Data":"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"}
Jul 24 20:08:13.332516 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.332463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs" event={"ID":"78861b5c-a5f8-47ac-967e-57833dbd7ce4","Type":"ContainerDied","Data":"43a7a98c1ab5802898dc6f5674a51059fabca5012cc5e4fef0c1b37a4f218fee"}
Jul 24 20:08:13.332516 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.332427    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"
Jul 24 20:08:13.339636 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.339615    2568 scope.go:117] "RemoveContainer" containerID="667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"
Jul 24 20:08:13.339928 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:08:13.339901    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d\": container with ID starting with 667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d not found: ID does not exist" containerID="667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"
Jul 24 20:08:13.339991 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.339939    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d"} err="failed to get container status \"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d\": rpc error: code = NotFound desc = could not find container \"667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d\": container with ID starting with 667177e74b5d14a671458616382bdc62e6942f2da6776ff206771dbeef3ca82d not found: ID does not exist"
Jul 24 20:08:13.339991 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.339958    2568 scope.go:117] "RemoveContainer" containerID="eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"
Jul 24 20:08:13.349032 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.349007    2568 scope.go:117] "RemoveContainer" containerID="eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"
Jul 24 20:08:13.349383 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:08:13.349352    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b\": container with ID starting with eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b not found: ID does not exist" containerID="eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"
Jul 24 20:08:13.349470 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.349382    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b"} err="failed to get container status \"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b\": rpc error: code = NotFound desc = could not find container \"eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b\": container with ID starting with eb5d4c735f8816044f2c92dc24b3970582d508cca1318ea3e7e03eb875e0ac9b not found: ID does not exist"
Jul 24 20:08:13.349470 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.349398    2568 scope.go:117] "RemoveContainer" containerID="47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"
Jul 24 20:08:13.357404 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.357376    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:08:13.358211 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.358189    2568 scope.go:117] "RemoveContainer" containerID="47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"
Jul 24 20:08:13.358543 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:08:13.358518    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4\": container with ID starting with 47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4 not found: ID does not exist" containerID="47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"
Jul 24 20:08:13.358639 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.358553    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4"} err="failed to get container status \"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4\": rpc error: code = NotFound desc = could not find container \"47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4\": container with ID starting with 47fac4008724f51bc570bcda9bf6cb38e0064cf92ca95deb2d656e7cae8a38f4 not found: ID does not exist"
Jul 24 20:08:13.362398 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.362375    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-74e2adbb-56f57fbf94-qm55l"]
Jul 24 20:08:13.372554 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.372529    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:08:13.376131 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.376109    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-74e2adbb-6f95487bd-cm7qs"]
Jul 24 20:08:13.388414 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.388386    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:08:13.390216 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.390197    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-74e2adbb-5ff959975c-4mbt9"]
Jul 24 20:08:13.431279 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.431236    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="488ff1ae-810d-4298-b92f-32337aae0fe6" path="/var/lib/kubelet/pods/488ff1ae-810d-4298-b92f-32337aae0fe6/volumes"
Jul 24 20:08:13.431700 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.431684    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="757e5c21-8e65-4136-aa93-c29387580f7a" path="/var/lib/kubelet/pods/757e5c21-8e65-4136-aa93-c29387580f7a/volumes"
Jul 24 20:08:13.432003 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:08:13.431991    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" path="/var/lib/kubelet/pods/78861b5c-a5f8-47ac-967e-57833dbd7ce4/volumes"
Jul 24 20:09:29.466243 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:09:29.466210    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:09:29.473191 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:09:29.473166    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:09:29.605116 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:09:29.605086    2568 scope.go:117] "RemoveContainer" containerID="46c30050c2a8529f7201fb6b85fafccfe52ce83cd11b451248b4b76e20f37ffe"
Jul 24 20:10:08.210812 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.210771    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:10:08.211469 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211412    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="488ff1ae-810d-4298-b92f-32337aae0fe6" containerName="payload-processing"
Jul 24 20:10:08.211469 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211436    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="488ff1ae-810d-4298-b92f-32337aae0fe6" containerName="payload-processing"
Jul 24 20:10:08.211469 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211469    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757e5c21-8e65-4136-aa93-c29387580f7a" containerName="maas-api"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211483    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="757e5c21-8e65-4136-aa93-c29387580f7a" containerName="maas-api"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211499    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerName="payload-pre-processing"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211507    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerName="payload-pre-processing"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211607    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="78861b5c-a5f8-47ac-967e-57833dbd7ce4" containerName="payload-pre-processing"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211624    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="757e5c21-8e65-4136-aa93-c29387580f7a" containerName="maas-api"
Jul 24 20:10:08.211667 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.211633    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="488ff1ae-810d-4298-b92f-32337aae0fe6" containerName="payload-processing"
Jul 24 20:10:08.214795 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.214775    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.217422 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.217401    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-05f5d4-serving-cert\""
Jul 24 20:10:08.227100 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.227072    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:10:08.300506 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.300472    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.300506 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.300510    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zks8g\" (UniqueName: \"kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.300731 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.300575    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.402023 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.401976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.402193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.402032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zks8g\" (UniqueName: \"kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.402193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.402083    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.402193 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:10:08.402139    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-serving-cert: secret "maas-api-e2e-inf-a-05f5d4-serving-cert" not found
Jul 24 20:10:08.402359 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:10:08.402221    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls podName:d2554af9-6767-4e66-bd59-75995b061bfb nodeName:}" failed. No retries permitted until 2026-07-24 20:10:08.902201104 +0000 UTC m=+2139.996746713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls") pod "maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" (UID: "d2554af9-6767-4e66-bd59-75995b061bfb") : secret "maas-api-e2e-inf-a-05f5d4-serving-cert" not found
Jul 24 20:10:08.402873 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.402850    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.411111 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.411086    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zks8g\" (UniqueName: \"kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.836028 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.835993    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:10:08.839991 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.839967    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:08.842878 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.842855    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-05f5d4\""
Jul 24 20:10:08.843061 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.842856    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-05f5d4-dockercfg-xfkqn\""
Jul 24 20:10:08.851507 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.851430    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:10:08.865641 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.865606    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:10:08.869543 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.869521    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:08.880508 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.880484    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:10:08.906591 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.906562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdwk9\" (UniqueName: \"kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:08.906769 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.906688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:08.906769 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.906720    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:08.909097 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:08.909071    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") pod \"maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:09.008149 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.008107    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.008380 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.008163    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q82s\" (UniqueName: \"kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.008380 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.008198    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdwk9\" (UniqueName: \"kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.008380 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.008308    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.008828 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.008807    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.016370 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.016350    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdwk9\" (UniqueName: \"kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9\") pod \"payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.109221 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.109120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q82s\" (UniqueName: \"kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.109221 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.109207    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.109931 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.109907    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.117754 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.117728    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q82s\" (UniqueName: \"kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s\") pod \"payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") " pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.127561 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.127540    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:09.153526 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.153378    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.183151 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.182853    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.295517 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.295485    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:10:09.297868 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:10:09.297821    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2554af9_6767_4e66_bd59_75995b061bfb.slice/crio-80dd3183a611d175f32f289a3fab65ecb30ca7cbf3043b63357a03f7dc0f0dbe WatchSource:0}: Error finding container 80dd3183a611d175f32f289a3fab65ecb30ca7cbf3043b63357a03f7dc0f0dbe: Status 404 returned error can't find the container with id 80dd3183a611d175f32f289a3fab65ecb30ca7cbf3043b63357a03f7dc0f0dbe
Jul 24 20:10:09.354248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.354220    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:10:09.355924 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:10:09.355890    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6383d560_5c71_4ec3_9532_cdb9eaf0bd23.slice/crio-97cf73295177e646ace98b437a74ebdec84b9b2f07f55d1be91cda938380b0ec WatchSource:0}: Error finding container 97cf73295177e646ace98b437a74ebdec84b9b2f07f55d1be91cda938380b0ec: Status 404 returned error can't find the container with id 97cf73295177e646ace98b437a74ebdec84b9b2f07f55d1be91cda938380b0ec
Jul 24 20:10:09.528789 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.528759    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:10:09.530760 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:10:09.530733    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3e2618_5aa1_4eba_a56d_721f09aa1c7e.slice/crio-dc8f8403fd8675cb8319e3916283c98a2bff9c6306366252228cfda34c67f630 WatchSource:0}: Error finding container dc8f8403fd8675cb8319e3916283c98a2bff9c6306366252228cfda34c67f630: Status 404 returned error can't find the container with id dc8f8403fd8675cb8319e3916283c98a2bff9c6306366252228cfda34c67f630
Jul 24 20:10:09.912192 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.912162    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" event={"ID":"6383d560-5c71-4ec3-9532-cdb9eaf0bd23","Type":"ContainerStarted","Data":"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"}
Jul 24 20:10:09.912384 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.912215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" event={"ID":"6383d560-5c71-4ec3-9532-cdb9eaf0bd23","Type":"ContainerStarted","Data":"97cf73295177e646ace98b437a74ebdec84b9b2f07f55d1be91cda938380b0ec"}
Jul 24 20:10:09.912384 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.912245    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:09.913754 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.913725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" event={"ID":"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e","Type":"ContainerStarted","Data":"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"}
Jul 24 20:10:09.913889 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.913778    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" event={"ID":"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e","Type":"ContainerStarted","Data":"dc8f8403fd8675cb8319e3916283c98a2bff9c6306366252228cfda34c67f630"}
Jul 24 20:10:09.913889 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.913812    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:10:09.915137 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.915104    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" event={"ID":"d2554af9-6767-4e66-bd59-75995b061bfb","Type":"ContainerStarted","Data":"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"}
Jul 24 20:10:09.915281 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.915149    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" event={"ID":"d2554af9-6767-4e66-bd59-75995b061bfb","Type":"ContainerStarted","Data":"80dd3183a611d175f32f289a3fab65ecb30ca7cbf3043b63357a03f7dc0f0dbe"}
Jul 24 20:10:09.915281 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.915174    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:09.928649 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.928610    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" podStartSLOduration=1.928594954 podStartE2EDuration="1.928594954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:10:09.927692878 +0000 UTC m=+2141.022238494" watchObservedRunningTime="2026-07-24 20:10:09.928594954 +0000 UTC m=+2141.023140557"
Jul 24 20:10:09.944073 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:09.944027    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" podStartSLOduration=1.450120015 podStartE2EDuration="1.944012266s" totalImagesPullingTime="493.892251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:10:09.943132731 +0000 UTC m=+2141.037678359" watchObservedRunningTime="2026-07-24 20:10:09.944012266 +0000 UTC m=+2141.038557878"
Jul 24 20:10:15.929813 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:15.929779    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:10:15.949341 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:15.949293    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" podStartSLOduration=7.949277851 podStartE2EDuration="7.949277851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:10:09.959495157 +0000 UTC m=+2141.054040771" watchObservedRunningTime="2026-07-24 20:10:15.949277851 +0000 UTC m=+2147.043823506"
Jul 24 20:10:16.118311 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.118279    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:10:16.122019 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.121999    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.125173 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.125139    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4c32d4-serving-cert\""
Jul 24 20:10:16.134012 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.133987    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:10:16.276020 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.275980    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.276020 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.276017    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.276322 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.276037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjns4\" (UniqueName: \"kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.376479 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.376439    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.376479 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.376478    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.376722 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.376497    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjns4\" (UniqueName: \"kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.376722 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:10:16.376608    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-serving-cert: secret "maas-api-e2e-inf-b-4c32d4-serving-cert" not found
Jul 24 20:10:16.376722 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:10:16.376683    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls podName:079d8d36-6302-4c9b-8888-45fcc9fc1e7c nodeName:}" failed. No retries permitted until 2026-07-24 20:10:16.87666322 +0000 UTC m=+2147.971208810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls") pod "maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" (UID: "079d8d36-6302-4c9b-8888-45fcc9fc1e7c") : secret "maas-api-e2e-inf-b-4c32d4-serving-cert" not found
Jul 24 20:10:16.377129 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.377109    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.386609 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.386579    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjns4\" (UniqueName: \"kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.881346 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.881308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:16.883979 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:16.883954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:17.035497 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.035439    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:17.181404 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.181368    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:10:17.182959 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:10:17.182924    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079d8d36_6302_4c9b_8888_45fcc9fc1e7c.slice/crio-0304ce875f6416c814a2f14d633ceb9bb88d87a290d7663f2c08685ed66df33b WatchSource:0}: Error finding container 0304ce875f6416c814a2f14d633ceb9bb88d87a290d7663f2c08685ed66df33b: Status 404 returned error can't find the container with id 0304ce875f6416c814a2f14d633ceb9bb88d87a290d7663f2c08685ed66df33b
Jul 24 20:10:17.961165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.961125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" event={"ID":"079d8d36-6302-4c9b-8888-45fcc9fc1e7c","Type":"ContainerStarted","Data":"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"}
Jul 24 20:10:17.961400 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.961191    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" event={"ID":"079d8d36-6302-4c9b-8888-45fcc9fc1e7c","Type":"ContainerStarted","Data":"0304ce875f6416c814a2f14d633ceb9bb88d87a290d7663f2c08685ed66df33b"}
Jul 24 20:10:17.961400 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.961343    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:17.978552 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:17.978500    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" podStartSLOduration=1.566745957 podStartE2EDuration="1.97847946s" totalImagesPullingTime="411.733503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:10:17.976997195 +0000 UTC m=+2149.071542809" watchObservedRunningTime="2026-07-24 20:10:17.97847946 +0000 UTC m=+2149.073025072"
Jul 24 20:10:20.924812 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:20.924766    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.71:9004: connect: connection refused"
Jul 24 20:10:20.926104 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:20.926074    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:10:23.971454 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:23.971424    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:10:30.924578 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:10:30.924538    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:11:41.980947 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:41.980891    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:11:41.981491 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:41.981176    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" podUID="d2554af9-6767-4e66-bd59-75995b061bfb" containerName="maas-api" containerID="cri-o://f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9" gracePeriod=30
Jul 24 20:11:42.032272 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.032224    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:11:42.032662 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.032475    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" podUID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" containerName="payload-processing" containerID="cri-o://98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a" gracePeriod=30
Jul 24 20:11:42.041055 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.040510    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:11:42.041055 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.040730    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerName="payload-pre-processing" containerID="cri-o://a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669" gracePeriod=30
Jul 24 20:11:42.316236 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.316210    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:11:42.345308 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.345252    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:11:42.351079 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.351057    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:11:42.388493 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.388453    2568 generic.go:356] "Generic (PLEG): container finished" podID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" containerID="98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a" exitCode=0
Jul 24 20:11:42.388715 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.388539    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"
Jul 24 20:11:42.388715 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.388575    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" event={"ID":"6383d560-5c71-4ec3-9532-cdb9eaf0bd23","Type":"ContainerDied","Data":"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"}
Jul 24 20:11:42.388715 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.388649    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj" event={"ID":"6383d560-5c71-4ec3-9532-cdb9eaf0bd23","Type":"ContainerDied","Data":"97cf73295177e646ace98b437a74ebdec84b9b2f07f55d1be91cda938380b0ec"}
Jul 24 20:11:42.388715 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.388683    2568 scope.go:117] "RemoveContainer" containerID="98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"
Jul 24 20:11:42.390036 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.390004    2568 generic.go:356] "Generic (PLEG): container finished" podID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerID="a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669" exitCode=0
Jul 24 20:11:42.390107 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.390043    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"
Jul 24 20:11:42.390159 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.390107    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" event={"ID":"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e","Type":"ContainerDied","Data":"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"}
Jul 24 20:11:42.390214 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.390195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf" event={"ID":"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e","Type":"ContainerDied","Data":"dc8f8403fd8675cb8319e3916283c98a2bff9c6306366252228cfda34c67f630"}
Jul 24 20:11:42.391556 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.391529    2568 generic.go:356] "Generic (PLEG): container finished" podID="d2554af9-6767-4e66-bd59-75995b061bfb" containerID="f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9" exitCode=0
Jul 24 20:11:42.391669 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.391570    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"
Jul 24 20:11:42.391669 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.391628    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" event={"ID":"d2554af9-6767-4e66-bd59-75995b061bfb","Type":"ContainerDied","Data":"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"}
Jul 24 20:11:42.391801 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.391698    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7" event={"ID":"d2554af9-6767-4e66-bd59-75995b061bfb","Type":"ContainerDied","Data":"80dd3183a611d175f32f289a3fab65ecb30ca7cbf3043b63357a03f7dc0f0dbe"}
Jul 24 20:11:42.401288 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.401239    2568 scope.go:117] "RemoveContainer" containerID="98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"
Jul 24 20:11:42.401612 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:11:42.401590    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a\": container with ID starting with 98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a not found: ID does not exist" containerID="98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"
Jul 24 20:11:42.401682 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.401622    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a"} err="failed to get container status \"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a\": rpc error: code = NotFound desc = could not find container \"98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a\": container with ID starting with 98f7ecb4211e233e2603e614c4c878f8118d5d89ce6edf3bbdbc64f551a5fc7a not found: ID does not exist"
Jul 24 20:11:42.401682 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.401640    2568 scope.go:117] "RemoveContainer" containerID="a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"
Jul 24 20:11:42.409679 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.409658    2568 scope.go:117] "RemoveContainer" containerID="a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"
Jul 24 20:11:42.409927 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:11:42.409904    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669\": container with ID starting with a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669 not found: ID does not exist" containerID="a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"
Jul 24 20:11:42.410004 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.409938    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669"} err="failed to get container status \"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669\": rpc error: code = NotFound desc = could not find container \"a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669\": container with ID starting with a7c6babddc22875f0cd8d979e26a378d1f66e82bd442ebacf9ac5defaa4e2669 not found: ID does not exist"
Jul 24 20:11:42.410004 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.409963    2568 scope.go:117] "RemoveContainer" containerID="f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"
Jul 24 20:11:42.417820 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.417803    2568 scope.go:117] "RemoveContainer" containerID="f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"
Jul 24 20:11:42.418080 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:11:42.418063    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9\": container with ID starting with f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9 not found: ID does not exist" containerID="f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"
Jul 24 20:11:42.418129 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.418084    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9"} err="failed to get container status \"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9\": rpc error: code = NotFound desc = could not find container \"f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9\": container with ID starting with f51e100f1a546972526933885b8f8dcbdc77304cfd3b4f6438f67bc084e036a9 not found: ID does not exist"
Jul 24 20:11:42.481551 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481516    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca\") pod \"d2554af9-6767-4e66-bd59-75995b061bfb\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") "
Jul 24 20:11:42.481709 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481597    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") pod \"d2554af9-6767-4e66-bd59-75995b061bfb\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") "
Jul 24 20:11:42.481709 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481633    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume\") pod \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") "
Jul 24 20:11:42.481709 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481690    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q82s\" (UniqueName: \"kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s\") pod \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\" (UID: \"6383d560-5c71-4ec3-9532-cdb9eaf0bd23\") "
Jul 24 20:11:42.481906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481734    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdwk9\" (UniqueName: \"kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9\") pod \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") "
Jul 24 20:11:42.481906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481761    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zks8g\" (UniqueName: \"kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g\") pod \"d2554af9-6767-4e66-bd59-75995b061bfb\" (UID: \"d2554af9-6767-4e66-bd59-75995b061bfb\") "
Jul 24 20:11:42.481906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.481783    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume\") pod \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\" (UID: \"ff3e2618-5aa1-4eba-a56d-721f09aa1c7e\") "
Jul 24 20:11:42.482067 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.482004    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2554af9-6767-4e66-bd59-75995b061bfb" (UID: "d2554af9-6767-4e66-bd59-75995b061bfb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:11:42.482128 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.482068    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2554af9-6767-4e66-bd59-75995b061bfb-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.482128 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.482090    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6383d560-5c71-4ec3-9532-cdb9eaf0bd23" (UID: "6383d560-5c71-4ec3-9532-cdb9eaf0bd23"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:11:42.482360 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.482327    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" (UID: "ff3e2618-5aa1-4eba-a56d-721f09aa1c7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:11:42.483800 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.483773    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2554af9-6767-4e66-bd59-75995b061bfb" (UID: "d2554af9-6767-4e66-bd59-75995b061bfb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:11:42.483998 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.483975    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9" (OuterVolumeSpecName: "kube-api-access-mdwk9") pod "ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" (UID: "ff3e2618-5aa1-4eba-a56d-721f09aa1c7e"). InnerVolumeSpecName "kube-api-access-mdwk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:11:42.484234 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.484217    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s" (OuterVolumeSpecName: "kube-api-access-5q82s") pod "6383d560-5c71-4ec3-9532-cdb9eaf0bd23" (UID: "6383d560-5c71-4ec3-9532-cdb9eaf0bd23"). InnerVolumeSpecName "kube-api-access-5q82s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:11:42.484317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.484247    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g" (OuterVolumeSpecName: "kube-api-access-zks8g") pod "d2554af9-6767-4e66-bd59-75995b061bfb" (UID: "d2554af9-6767-4e66-bd59-75995b061bfb"). InnerVolumeSpecName "kube-api-access-zks8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583109    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2554af9-6767-4e66-bd59-75995b061bfb-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583140    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583149    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q82s\" (UniqueName: \"kubernetes.io/projected/6383d560-5c71-4ec3-9532-cdb9eaf0bd23-kube-api-access-5q82s\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583160    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdwk9\" (UniqueName: \"kubernetes.io/projected/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-kube-api-access-mdwk9\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583169    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zks8g\" (UniqueName: \"kubernetes.io/projected/d2554af9-6767-4e66-bd59-75995b061bfb-kube-api-access-zks8g\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.583189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.583177    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:11:42.714999 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.714972    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:11:42.720305 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.720280    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-05f5d4-68fdf89dfd-tdzwj"]
Jul 24 20:11:42.730591 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.730567    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:11:42.734210 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.734190    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-05f5d4-5788fdb7bd-v9ztf"]
Jul 24 20:11:42.744916 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.744894    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:11:42.748420 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:42.748399    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-05f5d4-dc7b58db-684l7"]
Jul 24 20:11:43.430796 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:43.430721    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" path="/var/lib/kubelet/pods/6383d560-5c71-4ec3-9532-cdb9eaf0bd23/volumes"
Jul 24 20:11:43.431136 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:43.431100    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2554af9-6767-4e66-bd59-75995b061bfb" path="/var/lib/kubelet/pods/d2554af9-6767-4e66-bd59-75995b061bfb/volumes"
Jul 24 20:11:43.431426 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:11:43.431413    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" path="/var/lib/kubelet/pods/ff3e2618-5aa1-4eba-a56d-721f09aa1c7e/volumes"
Jul 24 20:12:03.953784 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:03.953680    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:12:03.954217 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:03.953946    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerName="maas-api" containerID="cri-o://2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788" gracePeriod=30
Jul 24 20:12:03.966823 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:03.966784    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.73:8443/health\": dial tcp 10.133.0.73:8443: connect: connection refused"
Jul 24 20:12:04.403523 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.403494    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:12:04.473704 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.473672    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca\") pod \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") "
Jul 24 20:12:04.473704 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.473709    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjns4\" (UniqueName: \"kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4\") pod \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") "
Jul 24 20:12:04.473916 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.473818    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") pod \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\" (UID: \"079d8d36-6302-4c9b-8888-45fcc9fc1e7c\") "
Jul 24 20:12:04.474127 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.474095    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "079d8d36-6302-4c9b-8888-45fcc9fc1e7c" (UID: "079d8d36-6302-4c9b-8888-45fcc9fc1e7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:12:04.475894 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.475870    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "079d8d36-6302-4c9b-8888-45fcc9fc1e7c" (UID: "079d8d36-6302-4c9b-8888-45fcc9fc1e7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:12:04.475988 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.475875    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4" (OuterVolumeSpecName: "kube-api-access-qjns4") pod "079d8d36-6302-4c9b-8888-45fcc9fc1e7c" (UID: "079d8d36-6302-4c9b-8888-45fcc9fc1e7c"). InnerVolumeSpecName "kube-api-access-qjns4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:12:04.507991 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.507909    2568 generic.go:356] "Generic (PLEG): container finished" podID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerID="2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788" exitCode=0
Jul 24 20:12:04.507991 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.507963    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"
Jul 24 20:12:04.508196 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.507991    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" event={"ID":"079d8d36-6302-4c9b-8888-45fcc9fc1e7c","Type":"ContainerDied","Data":"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"}
Jul 24 20:12:04.508196 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.508079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n" event={"ID":"079d8d36-6302-4c9b-8888-45fcc9fc1e7c","Type":"ContainerDied","Data":"0304ce875f6416c814a2f14d633ceb9bb88d87a290d7663f2c08685ed66df33b"}
Jul 24 20:12:04.508196 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.508110    2568 scope.go:117] "RemoveContainer" containerID="2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"
Jul 24 20:12:04.519394 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.519374    2568 scope.go:117] "RemoveContainer" containerID="2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"
Jul 24 20:12:04.519663 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:12:04.519641    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788\": container with ID starting with 2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788 not found: ID does not exist" containerID="2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"
Jul 24 20:12:04.519741 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.519673    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788"} err="failed to get container status \"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788\": rpc error: code = NotFound desc = could not find container \"2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788\": container with ID starting with 2219723348c786d9c571f2d23a4ff33dae21a562fec171f5091093a875ddc788 not found: ID does not exist"
Jul 24 20:12:04.533345 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.533319    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:12:04.537000 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.536980    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c32d4-5b67dc4cf-jlx4n"]
Jul 24 20:12:04.575498 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.575469    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:04.575498 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.575494    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:04.575651 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:04.575505    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjns4\" (UniqueName: \"kubernetes.io/projected/079d8d36-6302-4c9b-8888-45fcc9fc1e7c-kube-api-access-qjns4\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:05.430612 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:05.430576    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" path="/var/lib/kubelet/pods/079d8d36-6302-4c9b-8888-45fcc9fc1e7c/volumes"
Jul 24 20:12:19.829022 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.828989    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:19.829557 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829490    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" containerName="payload-processing"
Jul 24 20:12:19.829557 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829510    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" containerName="payload-processing"
Jul 24 20:12:19.829557 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829534    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerName="payload-pre-processing"
Jul 24 20:12:19.829557 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829544    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerName="payload-pre-processing"
Jul 24 20:12:19.829557 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829555    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerName="maas-api"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829565    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerName="maas-api"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829580    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2554af9-6767-4e66-bd59-75995b061bfb" containerName="maas-api"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829590    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2554af9-6767-4e66-bd59-75995b061bfb" containerName="maas-api"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829698    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="079d8d36-6302-4c9b-8888-45fcc9fc1e7c" containerName="maas-api"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829714    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6383d560-5c71-4ec3-9532-cdb9eaf0bd23" containerName="payload-processing"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829726    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3e2618-5aa1-4eba-a56d-721f09aa1c7e" containerName="payload-pre-processing"
Jul 24 20:12:19.829837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.829735    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2554af9-6767-4e66-bd59-75995b061bfb" containerName="maas-api"
Jul 24 20:12:19.834232 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.834207    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:19.836854 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.836832    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-3ff089-serving-cert\""
Jul 24 20:12:19.843907 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.843786    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:19.914804 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.914772    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:19.914966 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.914837    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frc6z\" (UniqueName: \"kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:19.915014 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:19.914961    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.015653 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.015622    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frc6z\" (UniqueName: \"kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.015829 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.015689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.015829 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.015742    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.015829 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:12:20.015799    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-serving-cert: secret "maas-api-e2e-api-a-3ff089-serving-cert" not found
Jul 24 20:12:20.015964 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:12:20.015883    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls podName:d6e8351d-d9fa-491f-80e2-b648590fa4e6 nodeName:}" failed. No retries permitted until 2026-07-24 20:12:20.515862509 +0000 UTC m=+2271.610408107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls") pod "maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" (UID: "d6e8351d-d9fa-491f-80e2-b648590fa4e6") : secret "maas-api-e2e-api-a-3ff089-serving-cert" not found
Jul 24 20:12:20.016470 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.016442    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.027673 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.027650    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frc6z\" (UniqueName: \"kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.521548 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.521505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.523895 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.523870    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") pod \"maas-api-e2e-api-a-3ff089-7f8b74c749-749cp\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.528896 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.528870    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:20.532592 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.532565    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.536027 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.535997    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-3ff089\""
Jul 24 20:12:20.558290 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.557913    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:20.570333 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.570051    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:20.574236 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.574214    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.575348 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.575324    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:20.577006 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.576983    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-3ff089-dockercfg-cmmrf\""
Jul 24 20:12:20.623188 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.623156    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.623351 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.623216    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwvv6\" (UniqueName: \"kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.723957 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.723919    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.724147 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.723971    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwvv6\" (UniqueName: \"kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.724147 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.724000    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.724147 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.724036    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj9gr\" (UniqueName: \"kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.724768 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.724733    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.734079 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.734045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwvv6\" (UniqueName: \"kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6\") pod \"payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.747860 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.747833    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:20.825116 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.825080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.825314 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.825125    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj9gr\" (UniqueName: \"kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.825807 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.825784    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.836643 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.836620    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj9gr\" (UniqueName: \"kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr\") pod \"payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") " pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.856594 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.856552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:20.884215 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.884179    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:20.900441 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.900414    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:20.904886 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:12:20.904786    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e8351d_d9fa_491f_80e2_b648590fa4e6.slice/crio-a384224a79ebab4d8410c2aba99bcb3d573861e9455a81a9e6690492db7ec51a WatchSource:0}: Error finding container a384224a79ebab4d8410c2aba99bcb3d573861e9455a81a9e6690492db7ec51a: Status 404 returned error can't find the container with id a384224a79ebab4d8410c2aba99bcb3d573861e9455a81a9e6690492db7ec51a
Jul 24 20:12:20.905628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:20.905608    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:12:21.018574 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.018547    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:21.020807 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:12:21.020779    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746eb670_d2c2_41a0_af66_e816e6263202.slice/crio-f515c83ff5d8f74192c7e672d063a62a4c46e793969aaca1c3a4c8b22f28dc2a WatchSource:0}: Error finding container f515c83ff5d8f74192c7e672d063a62a4c46e793969aaca1c3a4c8b22f28dc2a: Status 404 returned error can't find the container with id f515c83ff5d8f74192c7e672d063a62a4c46e793969aaca1c3a4c8b22f28dc2a
Jul 24 20:12:21.048635 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.048332    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:21.050325 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:12:21.050292    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1ed83ad_bbfd_491c_b9a6_8c934c50df9e.slice/crio-4b3b2811556f0b3803c615fd755fc8a94d5154e42d32dd3b35f2910cc2855fba WatchSource:0}: Error finding container 4b3b2811556f0b3803c615fd755fc8a94d5154e42d32dd3b35f2910cc2855fba: Status 404 returned error can't find the container with id 4b3b2811556f0b3803c615fd755fc8a94d5154e42d32dd3b35f2910cc2855fba
Jul 24 20:12:21.594307 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.594251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" event={"ID":"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e","Type":"ContainerStarted","Data":"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"}
Jul 24 20:12:21.594479 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.594337    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" event={"ID":"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e","Type":"ContainerStarted","Data":"4b3b2811556f0b3803c615fd755fc8a94d5154e42d32dd3b35f2910cc2855fba"}
Jul 24 20:12:21.594479 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.594373    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:21.595881 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.595851    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" event={"ID":"746eb670-d2c2-41a0-af66-e816e6263202","Type":"ContainerStarted","Data":"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"}
Jul 24 20:12:21.595995 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.595898    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:21.595995 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.595915    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" event={"ID":"746eb670-d2c2-41a0-af66-e816e6263202","Type":"ContainerStarted","Data":"f515c83ff5d8f74192c7e672d063a62a4c46e793969aaca1c3a4c8b22f28dc2a"}
Jul 24 20:12:21.597340 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.597317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" event={"ID":"d6e8351d-d9fa-491f-80e2-b648590fa4e6","Type":"ContainerStarted","Data":"2758bd77caad99a177cdbb63baa4ac2a84448c8097c60ef3f52c45e0164efc8f"}
Jul 24 20:12:21.597450 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.597354    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" event={"ID":"d6e8351d-d9fa-491f-80e2-b648590fa4e6","Type":"ContainerStarted","Data":"a384224a79ebab4d8410c2aba99bcb3d573861e9455a81a9e6690492db7ec51a"}
Jul 24 20:12:21.597450 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.597401    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:21.625739 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.625691    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" podStartSLOduration=1.6256766900000001 podStartE2EDuration="1.62567669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:12:21.623373692 +0000 UTC m=+2272.717919305" watchObservedRunningTime="2026-07-24 20:12:21.62567669 +0000 UTC m=+2272.720222302"
Jul 24 20:12:21.648303 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:21.648188    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" podStartSLOduration=2.158539784 podStartE2EDuration="2.648171028s" totalImagesPullingTime="489.631244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:12:21.646837921 +0000 UTC m=+2272.741383533" watchObservedRunningTime="2026-07-24 20:12:21.648171028 +0000 UTC m=+2272.742716642"
Jul 24 20:12:27.607825 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:27.607796    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:27.624950 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:27.624904    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" podStartSLOduration=7.62488906 podStartE2EDuration="7.62488906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:12:21.667091072 +0000 UTC m=+2272.761636686" watchObservedRunningTime="2026-07-24 20:12:27.62488906 +0000 UTC m=+2278.719434673"
Jul 24 20:12:28.088100 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.088058    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:12:28.092029 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.092007    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.095201 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.095145    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-247467-serving-cert\""
Jul 24 20:12:28.112302 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.112276    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:12:28.193115 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.193077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7rn\" (UniqueName: \"kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.193310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.193159    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.193310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.193195    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.294431 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.294395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.294604 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.294451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.294604 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.294477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7rn\" (UniqueName: \"kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.295177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.295144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.297027 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.297004    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.303939 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.303915    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7rn\" (UniqueName: \"kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn\") pod \"maas-api-e2e-api-b-247467-6f7484d68d-p5dh9\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.405287 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.405170    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:28.544815 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.544786    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:12:28.547717 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:12:28.547684    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8141e7_9fb1_463a_8d0c_4da4eff6bf91.slice/crio-494cea8cd42c8c20aefa4118ada5c00f150e11b9523d113d485e387fa0942a1e WatchSource:0}: Error finding container 494cea8cd42c8c20aefa4118ada5c00f150e11b9523d113d485e387fa0942a1e: Status 404 returned error can't find the container with id 494cea8cd42c8c20aefa4118ada5c00f150e11b9523d113d485e387fa0942a1e
Jul 24 20:12:28.632583 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:28.632503    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" event={"ID":"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91","Type":"ContainerStarted","Data":"494cea8cd42c8c20aefa4118ada5c00f150e11b9523d113d485e387fa0942a1e"}
Jul 24 20:12:29.639158 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:29.639115    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" event={"ID":"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91","Type":"ContainerStarted","Data":"5ccf8c445c76e065a7466395c1e196f16bb04992747e72905ea03d4970538458"}
Jul 24 20:12:29.639602 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:29.639239    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:29.657204 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:29.657141    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" podStartSLOduration=1.308196665 podStartE2EDuration="1.657123048s" totalImagesPullingTime="348.926383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:12:29.655804394 +0000 UTC m=+2280.750350020" watchObservedRunningTime="2026-07-24 20:12:29.657123048 +0000 UTC m=+2280.751668661"
Jul 24 20:12:32.604220 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:32.604183    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:32.604703 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:32.604324    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:35.649906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:35.649879    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:12:55.652415 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.652379    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:55.653195 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.653139    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" podUID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" containerName="maas-api" containerID="cri-o://2758bd77caad99a177cdbb63baa4ac2a84448c8097c60ef3f52c45e0164efc8f" gracePeriod=30
Jul 24 20:12:55.709693 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.709659    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:55.709897 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.709872    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" podUID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" containerName="payload-processing" containerID="cri-o://af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429" gracePeriod=30
Jul 24 20:12:55.715043 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.714987    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:55.715649 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.715440    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" podUID="746eb670-d2c2-41a0-af66-e816e6263202" containerName="payload-pre-processing" containerID="cri-o://282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338" gracePeriod=30
Jul 24 20:12:55.781837 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.781805    2568 generic.go:356] "Generic (PLEG): container finished" podID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" containerID="2758bd77caad99a177cdbb63baa4ac2a84448c8097c60ef3f52c45e0164efc8f" exitCode=0
Jul 24 20:12:55.782049 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.781855    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" event={"ID":"d6e8351d-d9fa-491f-80e2-b648590fa4e6","Type":"ContainerDied","Data":"2758bd77caad99a177cdbb63baa4ac2a84448c8097c60ef3f52c45e0164efc8f"}
Jul 24 20:12:55.974454 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:55.974424    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:56.015111 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.015084    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:56.018215 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.018193    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:56.047920 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.047888    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwvv6\" (UniqueName: \"kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6\") pod \"746eb670-d2c2-41a0-af66-e816e6263202\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") "
Jul 24 20:12:56.048047 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.047947    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume\") pod \"746eb670-d2c2-41a0-af66-e816e6263202\" (UID: \"746eb670-d2c2-41a0-af66-e816e6263202\") "
Jul 24 20:12:56.048047 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.047975    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume\") pod \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") "
Jul 24 20:12:56.048047 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.048011    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frc6z\" (UniqueName: \"kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z\") pod \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") "
Jul 24 20:12:56.048165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.048082    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca\") pod \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") "
Jul 24 20:12:56.048165 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.048142    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj9gr\" (UniqueName: \"kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr\") pod \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\" (UID: \"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e\") "
Jul 24 20:12:56.048228 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.048187    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") pod \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\" (UID: \"d6e8351d-d9fa-491f-80e2-b648590fa4e6\") "
Jul 24 20:12:56.049801 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.049191    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" (UID: "d1ed83ad-bbfd-491c-b9a6-8c934c50df9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:12:56.049801 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.049573    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6e8351d-d9fa-491f-80e2-b648590fa4e6" (UID: "d6e8351d-d9fa-491f-80e2-b648590fa4e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:12:56.049801 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.049684    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "746eb670-d2c2-41a0-af66-e816e6263202" (UID: "746eb670-d2c2-41a0-af66-e816e6263202"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:12:56.051277 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.051236    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6" (OuterVolumeSpecName: "kube-api-access-nwvv6") pod "746eb670-d2c2-41a0-af66-e816e6263202" (UID: "746eb670-d2c2-41a0-af66-e816e6263202"). InnerVolumeSpecName "kube-api-access-nwvv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:12:56.051563 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.051545    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6e8351d-d9fa-491f-80e2-b648590fa4e6" (UID: "d6e8351d-d9fa-491f-80e2-b648590fa4e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:12:56.051709 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.051685    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr" (OuterVolumeSpecName: "kube-api-access-mj9gr") pod "d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" (UID: "d1ed83ad-bbfd-491c-b9a6-8c934c50df9e"). InnerVolumeSpecName "kube-api-access-mj9gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:12:56.051921 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.051897    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z" (OuterVolumeSpecName: "kube-api-access-frc6z") pod "d6e8351d-d9fa-491f-80e2-b648590fa4e6" (UID: "d6e8351d-d9fa-491f-80e2-b648590fa4e6"). InnerVolumeSpecName "kube-api-access-frc6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:12:56.148937 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148902    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e8351d-d9fa-491f-80e2-b648590fa4e6-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.148937 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148933    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj9gr\" (UniqueName: \"kubernetes.io/projected/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-kube-api-access-mj9gr\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.148937 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148944    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e8351d-d9fa-491f-80e2-b648590fa4e6-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.149177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148954    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwvv6\" (UniqueName: \"kubernetes.io/projected/746eb670-d2c2-41a0-af66-e816e6263202-kube-api-access-nwvv6\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.149177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148963    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/746eb670-d2c2-41a0-af66-e816e6263202-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.149177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148976    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.149177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.148988    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frc6z\" (UniqueName: \"kubernetes.io/projected/d6e8351d-d9fa-491f-80e2-b648590fa4e6-kube-api-access-frc6z\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:12:56.789082 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.789047    2568 generic.go:356] "Generic (PLEG): container finished" podID="746eb670-d2c2-41a0-af66-e816e6263202" containerID="282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338" exitCode=0
Jul 24 20:12:56.789562 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.789089    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"
Jul 24 20:12:56.789562 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.789128    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" event={"ID":"746eb670-d2c2-41a0-af66-e816e6263202","Type":"ContainerDied","Data":"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"}
Jul 24 20:12:56.789562 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.789191    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59" event={"ID":"746eb670-d2c2-41a0-af66-e816e6263202","Type":"ContainerDied","Data":"f515c83ff5d8f74192c7e672d063a62a4c46e793969aaca1c3a4c8b22f28dc2a"}
Jul 24 20:12:56.789562 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.789218    2568 scope.go:117] "RemoveContainer" containerID="282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"
Jul 24 20:12:56.790607 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.790505    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"
Jul 24 20:12:56.790607 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.790538    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp" event={"ID":"d6e8351d-d9fa-491f-80e2-b648590fa4e6","Type":"ContainerDied","Data":"a384224a79ebab4d8410c2aba99bcb3d573861e9455a81a9e6690492db7ec51a"}
Jul 24 20:12:56.792109 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.792083    2568 generic.go:356] "Generic (PLEG): container finished" podID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" containerID="af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429" exitCode=0
Jul 24 20:12:56.792245 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.792122    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"
Jul 24 20:12:56.792245 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.792154    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" event={"ID":"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e","Type":"ContainerDied","Data":"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"}
Jul 24 20:12:56.792245 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.792193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj" event={"ID":"d1ed83ad-bbfd-491c-b9a6-8c934c50df9e","Type":"ContainerDied","Data":"4b3b2811556f0b3803c615fd755fc8a94d5154e42d32dd3b35f2910cc2855fba"}
Jul 24 20:12:56.800196 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.800175    2568 scope.go:117] "RemoveContainer" containerID="282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"
Jul 24 20:12:56.800521 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:12:56.800500    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338\": container with ID starting with 282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338 not found: ID does not exist" containerID="282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"
Jul 24 20:12:56.800628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.800528    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338"} err="failed to get container status \"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338\": rpc error: code = NotFound desc = could not find container \"282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338\": container with ID starting with 282ba8e56200334c1642c4a45b57c60088d0383e0b2a3502fedbf4ee6fcbd338 not found: ID does not exist"
Jul 24 20:12:56.800628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.800544    2568 scope.go:117] "RemoveContainer" containerID="2758bd77caad99a177cdbb63baa4ac2a84448c8097c60ef3f52c45e0164efc8f"
Jul 24 20:12:56.809169 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.809153    2568 scope.go:117] "RemoveContainer" containerID="af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"
Jul 24 20:12:56.815372 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.815350    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:56.817780 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.817765    2568 scope.go:117] "RemoveContainer" containerID="af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"
Jul 24 20:12:56.818047 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:12:56.818030    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429\": container with ID starting with af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429 not found: ID does not exist" containerID="af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"
Jul 24 20:12:56.818098 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.818053    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429"} err="failed to get container status \"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429\": rpc error: code = NotFound desc = could not find container \"af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429\": container with ID starting with af9341fe2b6fa29ca298f6757036bc553f7f976e4d46d7865daa2bbc32c6d429 not found: ID does not exist"
Jul 24 20:12:56.819803 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.819784    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ff089-7f8b74c749-749cp"]
Jul 24 20:12:56.829224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.829195    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:56.832701 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.832680    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-3ff089-65df9ff568-xhz59"]
Jul 24 20:12:56.842525 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.842505    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:56.846927 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:56.846907    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3ff089-67455cd8d8-8drzj"]
Jul 24 20:12:57.437513 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:57.437481    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="746eb670-d2c2-41a0-af66-e816e6263202" path="/var/lib/kubelet/pods/746eb670-d2c2-41a0-af66-e816e6263202/volumes"
Jul 24 20:12:57.437877 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:57.437861    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" path="/var/lib/kubelet/pods/d1ed83ad-bbfd-491c-b9a6-8c934c50df9e/volumes"
Jul 24 20:12:57.438239 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:12:57.438224    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" path="/var/lib/kubelet/pods/d6e8351d-d9fa-491f-80e2-b648590fa4e6/volumes"
Jul 24 20:13:22.686906 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:22.686807    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:13:22.687679 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:22.687063    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" podUID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" containerName="maas-api" containerID="cri-o://5ccf8c445c76e065a7466395c1e196f16bb04992747e72905ea03d4970538458" gracePeriod=30
Jul 24 20:13:22.920139 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:22.920104    2568 generic.go:356] "Generic (PLEG): container finished" podID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" containerID="5ccf8c445c76e065a7466395c1e196f16bb04992747e72905ea03d4970538458" exitCode=0
Jul 24 20:13:22.920306 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:22.920176    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" event={"ID":"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91","Type":"ContainerDied","Data":"5ccf8c445c76e065a7466395c1e196f16bb04992747e72905ea03d4970538458"}
Jul 24 20:13:22.937254 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:22.937189    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:13:23.099836 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.099801    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca\") pod \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") "
Jul 24 20:13:23.100002 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.099857    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls\") pod \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") "
Jul 24 20:13:23.100002 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.099897    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz7rn\" (UniqueName: \"kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn\") pod \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\" (UID: \"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91\") "
Jul 24 20:13:23.100180 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.100158    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" (UID: "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:13:23.101924 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.101899    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" (UID: "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:13:23.102019 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.101945    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn" (OuterVolumeSpecName: "kube-api-access-fz7rn") pod "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" (UID: "5c8141e7-9fb1-463a-8d0c-4da4eff6bf91"). InnerVolumeSpecName "kube-api-access-fz7rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:13:23.200738 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.200707    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:13:23.200738 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.200734    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:13:23.200738 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.200743    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz7rn\" (UniqueName: \"kubernetes.io/projected/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91-kube-api-access-fz7rn\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:13:23.925957 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.925921    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9" event={"ID":"5c8141e7-9fb1-463a-8d0c-4da4eff6bf91","Type":"ContainerDied","Data":"494cea8cd42c8c20aefa4118ada5c00f150e11b9523d113d485e387fa0942a1e"}
Jul 24 20:13:23.925957 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.925944    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"
Jul 24 20:13:23.926482 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.925994    2568 scope.go:117] "RemoveContainer" containerID="5ccf8c445c76e065a7466395c1e196f16bb04992747e72905ea03d4970538458"
Jul 24 20:13:23.944003 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.943978    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:13:23.948103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:23.948076    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-247467-6f7484d68d-p5dh9"]
Jul 24 20:13:25.430976 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:25.430928    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" path="/var/lib/kubelet/pods/5c8141e7-9fb1-463a-8d0c-4da4eff6bf91/volumes"
Jul 24 20:13:37.273782 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.273735    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:13:37.274374 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274355    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" containerName="maas-api"
Jul 24 20:13:37.274453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274377    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" containerName="maas-api"
Jul 24 20:13:37.274453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274394    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" containerName="payload-processing"
Jul 24 20:13:37.274453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274403    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" containerName="payload-processing"
Jul 24 20:13:37.274453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274433    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="746eb670-d2c2-41a0-af66-e816e6263202" containerName="payload-pre-processing"
Jul 24 20:13:37.274453 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274443    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="746eb670-d2c2-41a0-af66-e816e6263202" containerName="payload-pre-processing"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274456    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" containerName="maas-api"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274464    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" containerName="maas-api"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274552    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1ed83ad-bbfd-491c-b9a6-8c934c50df9e" containerName="payload-processing"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274562    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8141e7-9fb1-463a-8d0c-4da4eff6bf91" containerName="maas-api"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274573    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e8351d-d9fa-491f-80e2-b648590fa4e6" containerName="maas-api"
Jul 24 20:13:37.274656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.274581    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="746eb670-d2c2-41a0-af66-e816e6263202" containerName="payload-pre-processing"
Jul 24 20:13:37.277962 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.277940    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.280897 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.280879    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-53d0a0-serving-cert\""
Jul 24 20:13:37.289391 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.289058    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:13:37.319586 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.319552    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.319746 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.319610    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htwcj\" (UniqueName: \"kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.319746 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.319681    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.420352 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.420306    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htwcj\" (UniqueName: \"kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.420529 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.420374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.420529 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.420441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.420623 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:13:37.420552    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-serving-cert: secret "maas-api-e2e-shared-a-53d0a0-serving-cert" not found
Jul 24 20:13:37.420623 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:13:37.420619    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls podName:74a0f0de-fc1b-46d7-be2a-3a5f87c802da nodeName:}" failed. No retries permitted until 2026-07-24 20:13:37.920603191 +0000 UTC m=+2349.015148784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls") pod "maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" (UID: "74a0f0de-fc1b-46d7-be2a-3a5f87c802da") : secret "maas-api-e2e-shared-a-53d0a0-serving-cert" not found
Jul 24 20:13:37.421008 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.420987    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.428990 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.428966    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htwcj\" (UniqueName: \"kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.924047 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.924011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:37.924217 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:13:37.924172    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-serving-cert: secret "maas-api-e2e-shared-a-53d0a0-serving-cert" not found
Jul 24 20:13:37.924278 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:13:37.924240    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls podName:74a0f0de-fc1b-46d7-be2a-3a5f87c802da nodeName:}" failed. No retries permitted until 2026-07-24 20:13:38.924226327 +0000 UTC m=+2350.018771918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls") pod "maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" (UID: "74a0f0de-fc1b-46d7-be2a-3a5f87c802da") : secret "maas-api-e2e-shared-a-53d0a0-serving-cert" not found
Jul 24 20:13:37.999913 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:37.999881    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:13:38.003417 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.003396    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.006131 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.006110    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-53d0a0-dockercfg-4p5ct\""
Jul 24 20:13:38.006247 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.006149    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-53d0a0\""
Jul 24 20:13:38.014248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.014222    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:13:38.025248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.025217    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8vld\" (UniqueName: \"kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.025406 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.025361    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.043283 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.043245    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:13:38.046700 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.046685    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.059647 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.059621    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:13:38.126835 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.126796    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z67zn\" (UniqueName: \"kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.127029 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.126929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.127029 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.126988    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.127029 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.127019    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8vld\" (UniqueName: \"kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.127680 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.127656    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.135359 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.135335    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8vld\" (UniqueName: \"kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld\") pod \"payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") " pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.228472 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.228436    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z67zn\" (UniqueName: \"kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.228651 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.228540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.229193 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.229174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.237189 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.237158    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z67zn\" (UniqueName: \"kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn\") pod \"payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.316087 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.316048    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:38.360629 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.359760    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:38.478550 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.478476    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:13:38.480673 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:13:38.480645    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac29f63_5da8_4182_af48_dc0dd65fbc55.slice/crio-9baafd6a4d6ade7684dc5df35289def1171b16f7474f663ff549217f8c96310b WatchSource:0}: Error finding container 9baafd6a4d6ade7684dc5df35289def1171b16f7474f663ff549217f8c96310b: Status 404 returned error can't find the container with id 9baafd6a4d6ade7684dc5df35289def1171b16f7474f663ff549217f8c96310b
Jul 24 20:13:38.515019 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.514988    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:13:38.521461 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:13:38.521427    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6034a39e_8e89_4d2f_9015_52581aba7f5e.slice/crio-bee8be3648d5bee9de1f96db1f5c381a89ee457f26b333fa7c64b5121e346e60 WatchSource:0}: Error finding container bee8be3648d5bee9de1f96db1f5c381a89ee457f26b333fa7c64b5121e346e60: Status 404 returned error can't find the container with id bee8be3648d5bee9de1f96db1f5c381a89ee457f26b333fa7c64b5121e346e60
Jul 24 20:13:38.934800 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.934712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:38.937141 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:38.937114    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:39.004309 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.004250    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" event={"ID":"6034a39e-8e89-4d2f-9015-52581aba7f5e","Type":"ContainerStarted","Data":"ccb4dac8796e708695568b755c457b8ba49b128887ca1c64c33e1f3757653c7e"}
Jul 24 20:13:39.004498 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.004367    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" event={"ID":"6034a39e-8e89-4d2f-9015-52581aba7f5e","Type":"ContainerStarted","Data":"bee8be3648d5bee9de1f96db1f5c381a89ee457f26b333fa7c64b5121e346e60"}
Jul 24 20:13:39.004498 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.004429    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:39.005987 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.005960    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" event={"ID":"8ac29f63-5da8-4182-af48-dc0dd65fbc55","Type":"ContainerStarted","Data":"f0c738fc0d2eeea57cb5bfd2fc32aa47eea5ae2415ebbdbbc72f1739eab824eb"}
Jul 24 20:13:39.006099 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.006005    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:39.006099 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.006020    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" event={"ID":"8ac29f63-5da8-4182-af48-dc0dd65fbc55","Type":"ContainerStarted","Data":"9baafd6a4d6ade7684dc5df35289def1171b16f7474f663ff549217f8c96310b"}
Jul 24 20:13:39.022674 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.022631    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" podStartSLOduration=1.022616261 podStartE2EDuration="1.022616261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:13:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:13:39.021267628 +0000 UTC m=+2350.115813241" watchObservedRunningTime="2026-07-24 20:13:39.022616261 +0000 UTC m=+2350.117161873"
Jul 24 20:13:39.036599 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.036557    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" podStartSLOduration=2.036543866 podStartE2EDuration="2.036543866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:13:39.035905032 +0000 UTC m=+2350.130450651" watchObservedRunningTime="2026-07-24 20:13:39.036543866 +0000 UTC m=+2350.131089479"
Jul 24 20:13:39.094182 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.094148    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:39.239359 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:39.239330    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:13:39.241469 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:13:39.241424    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a0f0de_fc1b_46d7_be2a_3a5f87c802da.slice/crio-68d209ec5a403f543e4b7a6761745d493c26fcf476795c9c59fe281cc82cfb72 WatchSource:0}: Error finding container 68d209ec5a403f543e4b7a6761745d493c26fcf476795c9c59fe281cc82cfb72: Status 404 returned error can't find the container with id 68d209ec5a403f543e4b7a6761745d493c26fcf476795c9c59fe281cc82cfb72
Jul 24 20:13:40.011786 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:40.011748    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" event={"ID":"74a0f0de-fc1b-46d7-be2a-3a5f87c802da","Type":"ContainerStarted","Data":"68d209ec5a403f543e4b7a6761745d493c26fcf476795c9c59fe281cc82cfb72"}
Jul 24 20:13:41.019357 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:41.019317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" event={"ID":"74a0f0de-fc1b-46d7-be2a-3a5f87c802da","Type":"ContainerStarted","Data":"0512bf0f9eca2500a6baa720c6ef20207313e189eec9809e63f18bd8d35ee1d9"}
Jul 24 20:13:41.019787 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:41.019465    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:41.040333 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:41.040285    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" podStartSLOduration=3.194317957 podStartE2EDuration="4.04027051s" totalImagesPullingTime="845.952553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:13:41.037609331 +0000 UTC m=+2352.132154946" watchObservedRunningTime="2026-07-24 20:13:41.04027051 +0000 UTC m=+2352.134816132"
Jul 24 20:13:45.410104 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.410062    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:13:45.414125 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.414101    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.416997 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.416974    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-3f5fc9-serving-cert\""
Jul 24 20:13:45.434946 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.434916    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:13:45.493937 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.493891    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6xrw\" (UniqueName: \"kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.494089 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.494015    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.494089 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.494078    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.595459 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.595407    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.595648 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.595566    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.595808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.595769    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6xrw\" (UniqueName: \"kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.596561 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.596534    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.598464 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.598438    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.605191 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.605167    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6xrw\" (UniqueName: \"kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw\") pod \"maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.729462 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.729419    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:45.874823 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:45.874784    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:13:45.876516 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:13:45.876488    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5eb3ad_5628_4490_8b90_d5c12a9c7d04.slice/crio-e2fa8328088096b4b8adbada337c02dd779b920035de4bd7e020dd80e376ddc6 WatchSource:0}: Error finding container e2fa8328088096b4b8adbada337c02dd779b920035de4bd7e020dd80e376ddc6: Status 404 returned error can't find the container with id e2fa8328088096b4b8adbada337c02dd779b920035de4bd7e020dd80e376ddc6
Jul 24 20:13:46.044703 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:46.044613    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" event={"ID":"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04","Type":"ContainerStarted","Data":"e2fa8328088096b4b8adbada337c02dd779b920035de4bd7e020dd80e376ddc6"}
Jul 24 20:13:47.030112 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:47.030082    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:13:47.051898 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:47.051864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" event={"ID":"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04","Type":"ContainerStarted","Data":"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"}
Jul 24 20:13:47.052078 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:47.052000    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:13:47.070973 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:47.070921    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" podStartSLOduration=1.7247705340000001 podStartE2EDuration="2.070904592s" totalImagesPullingTime="346.134058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:13:47.069577806 +0000 UTC m=+2358.164123420" watchObservedRunningTime="2026-07-24 20:13:47.070904592 +0000 UTC m=+2358.165450205"
Jul 24 20:13:50.013880 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:50.013852    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:13:50.014351 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:50.014196    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:13:53.063548 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:13:53.063510    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:14:29.498978 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:14:29.498950    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:14:29.506361 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:14:29.506339    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:19:08.575011 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.574975    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140052728540336"]
Jul 24 20:19:08.578587 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.578566    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:08.581736 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.581714    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-z47rd\""
Jul 24 20:19:08.584559 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.584535    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728540336"]
Jul 24 20:19:08.629558 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.629531    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4lh\" (UniqueName: \"kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh\") pod \"test-curl-7494-140052728540336\" (UID: \"67617daa-ca46-48c0-904d-7f82ba7cde66\") " pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:08.731140 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.731095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4lh\" (UniqueName: \"kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh\") pod \"test-curl-7494-140052728540336\" (UID: \"67617daa-ca46-48c0-904d-7f82ba7cde66\") " pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:08.739214 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.739188    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4lh\" (UniqueName: \"kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh\") pod \"test-curl-7494-140052728540336\" (UID: \"67617daa-ca46-48c0-904d-7f82ba7cde66\") " pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:08.889491 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:08.889411    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:09.024863 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:09.024709    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728540336"]
Jul 24 20:19:09.027865 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:09.027837    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67617daa_ca46_48c0_904d_7f82ba7cde66.slice/crio-7959b2804fb5e3fc5ae6c3b99524763c451320ee8bbf7f64f5e182269d215391 WatchSource:0}: Error finding container 7959b2804fb5e3fc5ae6c3b99524763c451320ee8bbf7f64f5e182269d215391: Status 404 returned error can't find the container with id 7959b2804fb5e3fc5ae6c3b99524763c451320ee8bbf7f64f5e182269d215391
Jul 24 20:19:09.029568 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:09.029549    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:19:09.683277 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:09.683221    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728540336" event={"ID":"67617daa-ca46-48c0-904d-7f82ba7cde66","Type":"ContainerStarted","Data":"7959b2804fb5e3fc5ae6c3b99524763c451320ee8bbf7f64f5e182269d215391"}
Jul 24 20:19:10.689383 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:10.689350    2568 generic.go:356] "Generic (PLEG): container finished" podID="67617daa-ca46-48c0-904d-7f82ba7cde66" containerID="d9d8e4d145c03b54ebb313b6261b434fab183dcf064d9fe646bce5e1a3f71585" exitCode=0
Jul 24 20:19:10.689772 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:10.689446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728540336" event={"ID":"67617daa-ca46-48c0-904d-7f82ba7cde66","Type":"ContainerDied","Data":"d9d8e4d145c03b54ebb313b6261b434fab183dcf064d9fe646bce5e1a3f71585"}
Jul 24 20:19:11.835836 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.835811    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:11.870705 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.870680    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140052728540336_67617daa-ca46-48c0-904d-7f82ba7cde66/test-curl-7494-140052728540336/0.log"
Jul 24 20:19:11.892613 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.892581    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140052728540336"]
Jul 24 20:19:11.895842 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.895819    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140052728540336"]
Jul 24 20:19:11.960571 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.960496    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t4lh\" (UniqueName: \"kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh\") pod \"67617daa-ca46-48c0-904d-7f82ba7cde66\" (UID: \"67617daa-ca46-48c0-904d-7f82ba7cde66\") "
Jul 24 20:19:11.962475 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:11.962447    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh" (OuterVolumeSpecName: "kube-api-access-4t4lh") pod "67617daa-ca46-48c0-904d-7f82ba7cde66" (UID: "67617daa-ca46-48c0-904d-7f82ba7cde66"). InnerVolumeSpecName "kube-api-access-4t4lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:19:12.013267 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.013232    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140052728522672"]
Jul 24 20:19:12.013657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.013641    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67617daa-ca46-48c0-904d-7f82ba7cde66" containerName="test-curl-7494-140052728540336"
Jul 24 20:19:12.013657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.013658    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="67617daa-ca46-48c0-904d-7f82ba7cde66" containerName="test-curl-7494-140052728540336"
Jul 24 20:19:12.013771 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.013720    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="67617daa-ca46-48c0-904d-7f82ba7cde66" containerName="test-curl-7494-140052728540336"
Jul 24 20:19:12.016686 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.016670    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:12.023296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.023273    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728522672"]
Jul 24 20:19:12.061987 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.061958    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ppp\" (UniqueName: \"kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp\") pod \"test-curl-7494-140052728522672\" (UID: \"d8f2a4ce-1467-4e6e-8af0-894607f160a0\") " pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:12.062171 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.062008    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t4lh\" (UniqueName: \"kubernetes.io/projected/67617daa-ca46-48c0-904d-7f82ba7cde66-kube-api-access-4t4lh\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:19:12.162963 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.162922    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ppp\" (UniqueName: \"kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp\") pod \"test-curl-7494-140052728522672\" (UID: \"d8f2a4ce-1467-4e6e-8af0-894607f160a0\") " pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:12.171016 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.170985    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ppp\" (UniqueName: \"kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp\") pod \"test-curl-7494-140052728522672\" (UID: \"d8f2a4ce-1467-4e6e-8af0-894607f160a0\") " pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:12.328486 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.328384    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:12.464947 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.464913    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728522672"]
Jul 24 20:19:12.466353 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:12.466323    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8f2a4ce_1467_4e6e_8af0_894607f160a0.slice/crio-0b33108948bd122e88e6a053db14bae074024436ea20ac28dfee9e228e267372 WatchSource:0}: Error finding container 0b33108948bd122e88e6a053db14bae074024436ea20ac28dfee9e228e267372: Status 404 returned error can't find the container with id 0b33108948bd122e88e6a053db14bae074024436ea20ac28dfee9e228e267372
Jul 24 20:19:12.703709 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.703667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728522672" event={"ID":"d8f2a4ce-1467-4e6e-8af0-894607f160a0","Type":"ContainerStarted","Data":"0b33108948bd122e88e6a053db14bae074024436ea20ac28dfee9e228e267372"}
Jul 24 20:19:12.704732 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.704711    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7959b2804fb5e3fc5ae6c3b99524763c451320ee8bbf7f64f5e182269d215391"
Jul 24 20:19:12.704825 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:12.704751    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:12.722589 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:12.722560    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7494-140052728540336\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="67617daa-ca46-48c0-904d-7f82ba7cde66" pod="opendatahub/test-curl-7494-140052728540336"
Jul 24 20:19:13.431226 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:13.431192    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67617daa-ca46-48c0-904d-7f82ba7cde66" path="/var/lib/kubelet/pods/67617daa-ca46-48c0-904d-7f82ba7cde66/volumes"
Jul 24 20:19:13.711563 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:13.711523    2568 generic.go:356] "Generic (PLEG): container finished" podID="d8f2a4ce-1467-4e6e-8af0-894607f160a0" containerID="1708b5510916882a1fe6d1df765e20ab4ef61c64aa2222cbb1aab6c14ba18399" exitCode=0
Jul 24 20:19:13.711733 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:13.711612    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728522672" event={"ID":"d8f2a4ce-1467-4e6e-8af0-894607f160a0","Type":"ContainerDied","Data":"1708b5510916882a1fe6d1df765e20ab4ef61c64aa2222cbb1aab6c14ba18399"}
Jul 24 20:19:14.847820 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.847792    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:14.867022 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.866995    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140052728522672_d8f2a4ce-1467-4e6e-8af0-894607f160a0/test-curl-7494-140052728522672/0.log"
Jul 24 20:19:14.888704 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.888486    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2ppp\" (UniqueName: \"kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp\") pod \"d8f2a4ce-1467-4e6e-8af0-894607f160a0\" (UID: \"d8f2a4ce-1467-4e6e-8af0-894607f160a0\") "
Jul 24 20:19:14.891424 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.891397    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140052728522672"]
Jul 24 20:19:14.891762 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.891739    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp" (OuterVolumeSpecName: "kube-api-access-l2ppp") pod "d8f2a4ce-1467-4e6e-8af0-894607f160a0" (UID: "d8f2a4ce-1467-4e6e-8af0-894607f160a0"). InnerVolumeSpecName "kube-api-access-l2ppp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:19:14.893431 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.893412    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140052728522672"]
Jul 24 20:19:14.989360 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:14.989249    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2ppp\" (UniqueName: \"kubernetes.io/projected/d8f2a4ce-1467-4e6e-8af0-894607f160a0-kube-api-access-l2ppp\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:19:15.058884 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.058839    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:15.059245 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.059230    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f2a4ce-1467-4e6e-8af0-894607f160a0" containerName="test-curl-7494-140052728522672"
Jul 24 20:19:15.059245 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.059245    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f2a4ce-1467-4e6e-8af0-894607f160a0" containerName="test-curl-7494-140052728522672"
Jul 24 20:19:15.059407 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.059381    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f2a4ce-1467-4e6e-8af0-894607f160a0" containerName="test-curl-7494-140052728522672"
Jul 24 20:19:15.062071 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.062048    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:15.069340 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.069314    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:15.191225 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.191188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqkc\" (UniqueName: \"kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc\") pod \"test-curl-7494-140052724209392\" (UID: \"a4595c0c-2e77-446a-ab47-40dcf347ad59\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:15.292368 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.292278    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqkc\" (UniqueName: \"kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc\") pod \"test-curl-7494-140052724209392\" (UID: \"a4595c0c-2e77-446a-ab47-40dcf347ad59\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:15.300413 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.300379    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqkc\" (UniqueName: \"kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc\") pod \"test-curl-7494-140052724209392\" (UID: \"a4595c0c-2e77-446a-ab47-40dcf347ad59\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:15.373125 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.373085    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:15.436224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.436194    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f2a4ce-1467-4e6e-8af0-894607f160a0" path="/var/lib/kubelet/pods/d8f2a4ce-1467-4e6e-8af0-894607f160a0/volumes"
Jul 24 20:19:15.514427 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.514396    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:15.515971 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:15.515946    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4595c0c_2e77_446a_ab47_40dcf347ad59.slice/crio-583096b4e543f5266bd9f3a2128db9dc90dac9f384f783dba78d5451068163a5 WatchSource:0}: Error finding container 583096b4e543f5266bd9f3a2128db9dc90dac9f384f783dba78d5451068163a5: Status 404 returned error can't find the container with id 583096b4e543f5266bd9f3a2128db9dc90dac9f384f783dba78d5451068163a5
Jul 24 20:19:15.722444 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.722406    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052724209392" event={"ID":"a4595c0c-2e77-446a-ab47-40dcf347ad59","Type":"ContainerStarted","Data":"583096b4e543f5266bd9f3a2128db9dc90dac9f384f783dba78d5451068163a5"}
Jul 24 20:19:15.723639 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.723614    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728522672"
Jul 24 20:19:15.723825 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:15.723636    2568 scope.go:117] "RemoveContainer" containerID="1708b5510916882a1fe6d1df765e20ab4ef61c64aa2222cbb1aab6c14ba18399"
Jul 24 20:19:16.732248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:16.732214    2568 generic.go:356] "Generic (PLEG): container finished" podID="a4595c0c-2e77-446a-ab47-40dcf347ad59" containerID="4907345bd954120a98349f5ca49c1ff3e5123b962dfd4c6f395203068e02c064" exitCode=0
Jul 24 20:19:16.732639 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:16.732281    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052724209392" event={"ID":"a4595c0c-2e77-446a-ab47-40dcf347ad59","Type":"ContainerDied","Data":"4907345bd954120a98349f5ca49c1ff3e5123b962dfd4c6f395203068e02c064"}
Jul 24 20:19:17.897351 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:17.897326    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:17.916142 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:17.916118    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140052724209392_a4595c0c-2e77-446a-ab47-40dcf347ad59/test-curl-7494-140052724209392/0.log"
Jul 24 20:19:17.937891 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:17.937853    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:17.941494 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:17.941470    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:18.015143 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.015061    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grqkc\" (UniqueName: \"kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc\") pod \"a4595c0c-2e77-446a-ab47-40dcf347ad59\" (UID: \"a4595c0c-2e77-446a-ab47-40dcf347ad59\") "
Jul 24 20:19:18.017139 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.017107    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc" (OuterVolumeSpecName: "kube-api-access-grqkc") pod "a4595c0c-2e77-446a-ab47-40dcf347ad59" (UID: "a4595c0c-2e77-446a-ab47-40dcf347ad59"). InnerVolumeSpecName "kube-api-access-grqkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:19:18.058623 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.058588    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:18.059203 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.059183    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4595c0c-2e77-446a-ab47-40dcf347ad59" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:18.059296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.059206    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4595c0c-2e77-446a-ab47-40dcf347ad59" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:18.059352 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.059305    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4595c0c-2e77-446a-ab47-40dcf347ad59" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:18.062488 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.062467    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.066483 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.066457    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:18.116076 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.116042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gzdg\" (UniqueName: \"kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg\") pod \"test-curl-7494-140052724209392\" (UID: \"b4e213aa-d22a-478b-bc5b-d8bccff35712\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.116215 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.116145    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grqkc\" (UniqueName: \"kubernetes.io/projected/a4595c0c-2e77-446a-ab47-40dcf347ad59-kube-api-access-grqkc\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:19:18.217531 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.217489    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gzdg\" (UniqueName: \"kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg\") pod \"test-curl-7494-140052724209392\" (UID: \"b4e213aa-d22a-478b-bc5b-d8bccff35712\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.226003 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.225968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gzdg\" (UniqueName: \"kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg\") pod \"test-curl-7494-140052724209392\" (UID: \"b4e213aa-d22a-478b-bc5b-d8bccff35712\") " pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.373877 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.373793    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.511232 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.509741    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:18.512648 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:18.512616    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4e213aa_d22a_478b_bc5b_d8bccff35712.slice/crio-3edde2894b0d722e92d708665af6e78753e2e25eaa689795ab247cecccd2029a WatchSource:0}: Error finding container 3edde2894b0d722e92d708665af6e78753e2e25eaa689795ab247cecccd2029a: Status 404 returned error can't find the container with id 3edde2894b0d722e92d708665af6e78753e2e25eaa689795ab247cecccd2029a
Jul 24 20:19:18.742655 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.742625    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:18.742655 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.742655    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="583096b4e543f5266bd9f3a2128db9dc90dac9f384f783dba78d5451068163a5"
Jul 24 20:19:18.743830 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.743803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052724209392" event={"ID":"b4e213aa-d22a-478b-bc5b-d8bccff35712","Type":"ContainerStarted","Data":"3edde2894b0d722e92d708665af6e78753e2e25eaa689795ab247cecccd2029a"}
Jul 24 20:19:18.754651 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:18.754622    2568 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7494-140052724209392" oldPodUID="a4595c0c-2e77-446a-ab47-40dcf347ad59" podUID="b4e213aa-d22a-478b-bc5b-d8bccff35712"
Jul 24 20:19:19.431221 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:19.431187    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4595c0c-2e77-446a-ab47-40dcf347ad59" path="/var/lib/kubelet/pods/a4595c0c-2e77-446a-ab47-40dcf347ad59/volumes"
Jul 24 20:19:19.750305 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:19.750252    2568 generic.go:356] "Generic (PLEG): container finished" podID="b4e213aa-d22a-478b-bc5b-d8bccff35712" containerID="f69ce5de1f87a1b890e5e6c0598e94bfe090f25a05abb8baa5f66451ea3bd787" exitCode=0
Jul 24 20:19:19.750305 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:19.750299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052724209392" event={"ID":"b4e213aa-d22a-478b-bc5b-d8bccff35712","Type":"ContainerDied","Data":"f69ce5de1f87a1b890e5e6c0598e94bfe090f25a05abb8baa5f66451ea3bd787"}
Jul 24 20:19:20.903281 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:20.903228    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:20.924297 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:20.924253    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140052724209392_b4e213aa-d22a-478b-bc5b-d8bccff35712/test-curl-7494-140052724209392/0.log"
Jul 24 20:19:20.947670 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:20.947634    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:20.951311 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:20.951285    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140052724209392"]
Jul 24 20:19:21.041679 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.041558    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gzdg\" (UniqueName: \"kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg\") pod \"b4e213aa-d22a-478b-bc5b-d8bccff35712\" (UID: \"b4e213aa-d22a-478b-bc5b-d8bccff35712\") "
Jul 24 20:19:21.043617 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.043588    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg" (OuterVolumeSpecName: "kube-api-access-5gzdg") pod "b4e213aa-d22a-478b-bc5b-d8bccff35712" (UID: "b4e213aa-d22a-478b-bc5b-d8bccff35712"). InnerVolumeSpecName "kube-api-access-5gzdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:19:21.063634 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.063604    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140052728624592"]
Jul 24 20:19:21.064018 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.064005    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e213aa-d22a-478b-bc5b-d8bccff35712" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:21.064071 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.064020    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e213aa-d22a-478b-bc5b-d8bccff35712" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:21.064111 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.064081    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e213aa-d22a-478b-bc5b-d8bccff35712" containerName="test-curl-7494-140052724209392"
Jul 24 20:19:21.067137 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.067119    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:21.107782 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.107752    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728624592"]
Jul 24 20:19:21.142541 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.142505    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjt5r\" (UniqueName: \"kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r\") pod \"test-curl-7494-140052728624592\" (UID: \"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c\") " pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:21.142732 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.142714    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gzdg\" (UniqueName: \"kubernetes.io/projected/b4e213aa-d22a-478b-bc5b-d8bccff35712-kube-api-access-5gzdg\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:19:21.244097 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.244060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjt5r\" (UniqueName: \"kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r\") pod \"test-curl-7494-140052728624592\" (UID: \"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c\") " pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:21.251995 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.251965    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjt5r\" (UniqueName: \"kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r\") pod \"test-curl-7494-140052728624592\" (UID: \"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c\") " pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:21.376657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.376560    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:21.432832 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.432802    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e213aa-d22a-478b-bc5b-d8bccff35712" path="/var/lib/kubelet/pods/b4e213aa-d22a-478b-bc5b-d8bccff35712/volumes"
Jul 24 20:19:21.514716 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.514691    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140052728624592"]
Jul 24 20:19:21.515898 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:21.515871    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbc9ae8c_d862_4c64_ba70_7e5450c75a3c.slice/crio-a728159b05eb1481140efe0208249a5bf57bdafa0ecd163899c2353a47e7c9a6 WatchSource:0}: Error finding container a728159b05eb1481140efe0208249a5bf57bdafa0ecd163899c2353a47e7c9a6: Status 404 returned error can't find the container with id a728159b05eb1481140efe0208249a5bf57bdafa0ecd163899c2353a47e7c9a6
Jul 24 20:19:21.761019 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.760981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728624592" event={"ID":"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c","Type":"ContainerStarted","Data":"a728159b05eb1481140efe0208249a5bf57bdafa0ecd163899c2353a47e7c9a6"}
Jul 24 20:19:21.762248 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.762228    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052724209392"
Jul 24 20:19:21.762427 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:21.762250    2568 scope.go:117] "RemoveContainer" containerID="f69ce5de1f87a1b890e5e6c0598e94bfe090f25a05abb8baa5f66451ea3bd787"
Jul 24 20:19:22.775432 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:22.775394    2568 generic.go:356] "Generic (PLEG): container finished" podID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" containerID="55b0fc51516225f03275fd5657b95a5df6dd8aa503bf91b789e3b3da6a1493cb" exitCode=0
Jul 24 20:19:22.775866 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:22.775483    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140052728624592" event={"ID":"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c","Type":"ContainerDied","Data":"55b0fc51516225f03275fd5657b95a5df6dd8aa503bf91b789e3b3da6a1493cb"}
Jul 24 20:19:23.913910 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.913884    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:23.934772 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.934749    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140052728624592_dbc9ae8c-d862-4c64-ba70-7e5450c75a3c/test-curl-7494-140052728624592/0.log"
Jul 24 20:19:23.959315 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.959288    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140052728624592"]
Jul 24 20:19:23.961444 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.961419    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140052728624592"]
Jul 24 20:19:23.967739 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.967713    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjt5r\" (UniqueName: \"kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r\") pod \"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c\" (UID: \"dbc9ae8c-d862-4c64-ba70-7e5450c75a3c\") "
Jul 24 20:19:23.969708 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:23.969682    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r" (OuterVolumeSpecName: "kube-api-access-jjt5r") pod "dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" (UID: "dbc9ae8c-d862-4c64-ba70-7e5450c75a3c"). InnerVolumeSpecName "kube-api-access-jjt5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:19:24.069034 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:24.068947    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjt5r\" (UniqueName: \"kubernetes.io/projected/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c-kube-api-access-jjt5r\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:19:24.787163 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:24.787134    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:24.787342 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:24.787141    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a728159b05eb1481140efe0208249a5bf57bdafa0ecd163899c2353a47e7c9a6"
Jul 24 20:19:24.797076 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:24.797050    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7494-140052728624592\" is forbidden: User \"system:node:ip-10-0-129-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-222.ec2.internal' and this object" podUID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" pod="opendatahub/test-curl-7494-140052728624592"
Jul 24 20:19:25.431033 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:25.430998    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" path="/var/lib/kubelet/pods/dbc9ae8c-d862-4c64-ba70-7e5450c75a3c/volumes"
Jul 24 20:19:29.533020 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:29.532983    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:19:29.542538 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:29.542517    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:19:31.176851 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.176813    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:19:31.177463 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.177445    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" containerName="test-curl-7494-140052728624592"
Jul 24 20:19:31.177519 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.177466    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" containerName="test-curl-7494-140052728624592"
Jul 24 20:19:31.177589 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.177576    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc9ae8c-d862-4c64-ba70-7e5450c75a3c" containerName="test-curl-7494-140052728624592"
Jul 24 20:19:31.182508 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.182484    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.185086 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.185064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-937547-serving-cert\""
Jul 24 20:19:31.191999 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.191970    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:19:31.341467 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.341430    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5r56\" (UniqueName: \"kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.341668 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.341482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.341668 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.341614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.442810 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.442787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5r56\" (UniqueName: \"kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.442969 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.442832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.442969 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.442874    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.443052 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:31.443004    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-serving-cert: secret "maas-api-e2e-ipp-a-937547-serving-cert" not found
Jul 24 20:19:31.443108 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:31.443096    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls podName:4737673a-a61d-4b68-83ad-548b4668b91a nodeName:}" failed. No retries permitted until 2026-07-24 20:19:31.943063476 +0000 UTC m=+2703.037609084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls") pod "maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" (UID: "4737673a-a61d-4b68-83ad-548b4668b91a") : secret "maas-api-e2e-ipp-a-937547-serving-cert" not found
Jul 24 20:19:31.443612 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.443593    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.453134 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.453103    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5r56\" (UniqueName: \"kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.948021 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.947988    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:31.950606 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:31.950579    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:32.094904 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:32.094868    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:32.249726 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:32.249685    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:19:32.251423 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:32.251370    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4737673a_a61d_4b68_83ad_548b4668b91a.slice/crio-b6f26eca8861a0d024afe9ee98b5dc9c0a7e2fabc7f9a1ddba8605eeb209e630 WatchSource:0}: Error finding container b6f26eca8861a0d024afe9ee98b5dc9c0a7e2fabc7f9a1ddba8605eeb209e630: Status 404 returned error can't find the container with id b6f26eca8861a0d024afe9ee98b5dc9c0a7e2fabc7f9a1ddba8605eeb209e630
Jul 24 20:19:32.829738 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:32.829696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" event={"ID":"4737673a-a61d-4b68-83ad-548b4668b91a","Type":"ContainerStarted","Data":"b6f26eca8861a0d024afe9ee98b5dc9c0a7e2fabc7f9a1ddba8605eeb209e630"}
Jul 24 20:19:33.837341 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:33.837299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" event={"ID":"4737673a-a61d-4b68-83ad-548b4668b91a","Type":"ContainerStarted","Data":"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"}
Jul 24 20:19:33.837758 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:33.837402    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:33.857026 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:33.856968    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" podStartSLOduration=2.214695538 podStartE2EDuration="2.856948072s" totalImagesPullingTime="642.252534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:19:33.854068938 +0000 UTC m=+2704.948614552" watchObservedRunningTime="2026-07-24 20:19:33.856948072 +0000 UTC m=+2704.951493686"
Jul 24 20:19:39.848647 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:39.848614    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:19:50.294219 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.294183    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:19:50.300282 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.300247    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.303058 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.303033    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-307a30-serving-cert\""
Jul 24 20:19:50.311354 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.311326    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:19:50.317661 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.317595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjx5l\" (UniqueName: \"kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.317800 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.317705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.317800 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.317754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.418468 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.418430    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjx5l\" (UniqueName: \"kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.418656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.418490    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.418656 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.418529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.418734 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:50.418661    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-serving-cert: secret "maas-api-e2e-ipp-b-307a30-serving-cert" not found
Jul 24 20:19:50.418734 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:50.418730    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls podName:96473a99-2788-4678-aaf0-0f45d0eaa97f nodeName:}" failed. No retries permitted until 2026-07-24 20:19:50.9187111 +0000 UTC m=+2722.013256695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls") pod "maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" (UID: "96473a99-2788-4678-aaf0-0f45d0eaa97f") : secret "maas-api-e2e-ipp-b-307a30-serving-cert" not found
Jul 24 20:19:50.419241 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.419220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.434716 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.434689    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjx5l\" (UniqueName: \"kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.924058 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:50.924008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:50.924239 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:50.924170    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-serving-cert: secret "maas-api-e2e-ipp-b-307a30-serving-cert" not found
Jul 24 20:19:50.924328 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:19:50.924277    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls podName:96473a99-2788-4678-aaf0-0f45d0eaa97f nodeName:}" failed. No retries permitted until 2026-07-24 20:19:51.924232687 +0000 UTC m=+2723.018778282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls") pod "maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" (UID: "96473a99-2788-4678-aaf0-0f45d0eaa97f") : secret "maas-api-e2e-ipp-b-307a30-serving-cert" not found
Jul 24 20:19:51.104568 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.104525    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:19:51.111004 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.110970    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.114018 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.113990    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-307a30\""
Jul 24 20:19:51.114246 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.114229    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-307a30-dockercfg-gf8vg\""
Jul 24 20:19:51.115433 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.115394    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:19:51.225862 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.225832    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.225862 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.225866    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnr2h\" (UniqueName: \"kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.327087 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.327051    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.327087 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.327095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnr2h\" (UniqueName: \"kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.327854 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.327820    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.335624 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.335602    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnr2h\" (UniqueName: \"kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h\") pod \"payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.425119 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.425081    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.565031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.564995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:19:51.566398 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:51.566370    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabfcf0c5_5887_42d8_8e5b_57ec483b7205.slice/crio-c16a6dae0b93ed8eec43f8e736b41a5675c0a8eadbf457cd9b0fc8593856afe9 WatchSource:0}: Error finding container c16a6dae0b93ed8eec43f8e736b41a5675c0a8eadbf457cd9b0fc8593856afe9: Status 404 returned error can't find the container with id c16a6dae0b93ed8eec43f8e736b41a5675c0a8eadbf457cd9b0fc8593856afe9
Jul 24 20:19:51.932924 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.932824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:51.935428 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.935403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:51.943423 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.943393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" event={"ID":"abfcf0c5-5887-42d8-8e5b-57ec483b7205","Type":"ContainerStarted","Data":"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"}
Jul 24 20:19:51.943579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.943447    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" event={"ID":"abfcf0c5-5887-42d8-8e5b-57ec483b7205","Type":"ContainerStarted","Data":"c16a6dae0b93ed8eec43f8e736b41a5675c0a8eadbf457cd9b0fc8593856afe9"}
Jul 24 20:19:51.943579 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.943472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:19:51.960436 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:51.960392    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" podStartSLOduration=0.960378378 podStartE2EDuration="960.378378ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:19:51.958636657 +0000 UTC m=+2723.053182272" watchObservedRunningTime="2026-07-24 20:19:51.960378378 +0000 UTC m=+2723.054924036"
Jul 24 20:19:52.114952 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.114908    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:52.267001 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.266974    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:19:52.269291 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:19:52.269236    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96473a99_2788_4678_aaf0_0f45d0eaa97f.slice/crio-61e089b9faa93bf46d0d524dc143b941020a630a7a193eeaebccbb9b2210b8cc WatchSource:0}: Error finding container 61e089b9faa93bf46d0d524dc143b941020a630a7a193eeaebccbb9b2210b8cc: Status 404 returned error can't find the container with id 61e089b9faa93bf46d0d524dc143b941020a630a7a193eeaebccbb9b2210b8cc
Jul 24 20:19:52.950364 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.950322    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" event={"ID":"96473a99-2788-4678-aaf0-0f45d0eaa97f","Type":"ContainerStarted","Data":"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"}
Jul 24 20:19:52.950746 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.950382    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" event={"ID":"96473a99-2788-4678-aaf0-0f45d0eaa97f","Type":"ContainerStarted","Data":"61e089b9faa93bf46d0d524dc143b941020a630a7a193eeaebccbb9b2210b8cc"}
Jul 24 20:19:52.950746 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.950408    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:19:52.972221 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:52.972171    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" podStartSLOduration=2.605085085 podStartE2EDuration="2.972154221s" totalImagesPullingTime="367.069136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:19:52.969560544 +0000 UTC m=+2724.064106158" watchObservedRunningTime="2026-07-24 20:19:52.972154221 +0000 UTC m=+2724.066699834"
Jul 24 20:19:58.961927 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:19:58.961896    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:20:02.953376 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:02.953340    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:20:11.856755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:11.856718    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb_abfcf0c5-5887-42d8-8e5b-57ec483b7205/payload-processing/0.log"
Jul 24 20:20:53.672153 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:53.672080    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb_abfcf0c5-5887-42d8-8e5b-57ec483b7205/payload-processing/0.log"
Jul 24 20:20:57.556224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.556184    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:20:57.560188 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.560161    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.563783 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.563757    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-712eee-serving-cert\""
Jul 24 20:20:57.574517 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.573035    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:20:57.648271 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.648224    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcsdr\" (UniqueName: \"kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.648463 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.648376    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.648463 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.648432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.749635 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.749602    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.749808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.749649    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.749808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.749712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcsdr\" (UniqueName: \"kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.750436 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.750404    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.752034 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.752010    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.768175 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.768143    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcsdr\" (UniqueName: \"kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr\") pod \"maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:57.874788 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:57.874691    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:58.022996 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.022931    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:20:58.025498 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:20:58.025471    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04cdbb97_5fc3_484e_a3e6_1f0f0a208272.slice/crio-c25d5e6343059da253f9be3deb658b12b2f1aa4184d2654e881560beb21ebb9b WatchSource:0}: Error finding container c25d5e6343059da253f9be3deb658b12b2f1aa4184d2654e881560beb21ebb9b: Status 404 returned error can't find the container with id c25d5e6343059da253f9be3deb658b12b2f1aa4184d2654e881560beb21ebb9b
Jul 24 20:20:58.291748 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.291716    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" event={"ID":"04cdbb97-5fc3-484e-a3e6-1f0f0a208272","Type":"ContainerStarted","Data":"c25d5e6343059da253f9be3deb658b12b2f1aa4184d2654e881560beb21ebb9b"}
Jul 24 20:20:58.484720 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.484684    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:20:58.488331 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.488307    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.491134 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.491109    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-712eee-dockercfg-jjx5z\""
Jul 24 20:20:58.491279 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.491109    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-712eee\""
Jul 24 20:20:58.497702 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.497626    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:20:58.557214 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.557119    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.557669 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.557346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmp6r\" (UniqueName: \"kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.658302 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.658248    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.658492 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.658354    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmp6r\" (UniqueName: \"kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.659373 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.659341    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.666812 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.666780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmp6r\" (UniqueName: \"kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r\") pod \"payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.800835 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.800797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:58.956061 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:58.956030    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:20:58.959230 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:20:58.959199    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a7961e_30fa_4382_ab25_5a804afca040.slice/crio-6c94420b169b445378d3be0b4b0cc98b9da09dc80981ea8971105f68f75a5c64 WatchSource:0}: Error finding container 6c94420b169b445378d3be0b4b0cc98b9da09dc80981ea8971105f68f75a5c64: Status 404 returned error can't find the container with id 6c94420b169b445378d3be0b4b0cc98b9da09dc80981ea8971105f68f75a5c64
Jul 24 20:20:59.300319 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.300283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" event={"ID":"24a7961e-30fa-4382-ab25-5a804afca040","Type":"ContainerStarted","Data":"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"}
Jul 24 20:20:59.300491 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.300342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" event={"ID":"24a7961e-30fa-4382-ab25-5a804afca040","Type":"ContainerStarted","Data":"6c94420b169b445378d3be0b4b0cc98b9da09dc80981ea8971105f68f75a5c64"}
Jul 24 20:20:59.300617 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.300483    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:20:59.302094 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.302070    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" event={"ID":"04cdbb97-5fc3-484e-a3e6-1f0f0a208272","Type":"ContainerStarted","Data":"2a60b3480547822c4a1c7d3b8ff80a5777206f8c291cc6f37823528745982173"}
Jul 24 20:20:59.302210 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.302133    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:20:59.317660 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.317610    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" podStartSLOduration=1.317591245 podStartE2EDuration="1.317591245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:20:59.316027251 +0000 UTC m=+2790.410572875" watchObservedRunningTime="2026-07-24 20:20:59.317591245 +0000 UTC m=+2790.412136909"
Jul 24 20:20:59.332032 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:20:59.331983    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" podStartSLOduration=1.9562755269999998 podStartE2EDuration="2.331970134s" totalImagesPullingTime="375.694607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:20:59.331905574 +0000 UTC m=+2790.426451200" watchObservedRunningTime="2026-07-24 20:20:59.331970134 +0000 UTC m=+2790.426515750"
Jul 24 20:21:05.313786 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:05.313755    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:21:10.310090 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:10.310057    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:21:23.300786 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.300748    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:21:23.301363 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.301064    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" podUID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" containerName="maas-api" containerID="cri-o://2a60b3480547822c4a1c7d3b8ff80a5777206f8c291cc6f37823528745982173" gracePeriod=30
Jul 24 20:21:23.380388 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.380356    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:21:23.380620 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.380595    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" podUID="24a7961e-30fa-4382-ab25-5a804afca040" containerName="payload-processing" containerID="cri-o://d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a" gracePeriod=30
Jul 24 20:21:23.442012 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.441980    2568 generic.go:356] "Generic (PLEG): container finished" podID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" containerID="2a60b3480547822c4a1c7d3b8ff80a5777206f8c291cc6f37823528745982173" exitCode=0
Jul 24 20:21:23.442170 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.442063    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" event={"ID":"04cdbb97-5fc3-484e-a3e6-1f0f0a208272","Type":"ContainerDied","Data":"2a60b3480547822c4a1c7d3b8ff80a5777206f8c291cc6f37823528745982173"}
Jul 24 20:21:23.634515 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.634484    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:21:23.669892 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.669867    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:21:23.691584 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.691548    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcsdr\" (UniqueName: \"kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr\") pod \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") "
Jul 24 20:21:23.691755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.691620    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls\") pod \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") "
Jul 24 20:21:23.691755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.691654    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca\") pod \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\" (UID: \"04cdbb97-5fc3-484e-a3e6-1f0f0a208272\") "
Jul 24 20:21:23.691755 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.691704    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume\") pod \"24a7961e-30fa-4382-ab25-5a804afca040\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") "
Jul 24 20:21:23.691889 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.691753    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmp6r\" (UniqueName: \"kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r\") pod \"24a7961e-30fa-4382-ab25-5a804afca040\" (UID: \"24a7961e-30fa-4382-ab25-5a804afca040\") "
Jul 24 20:21:23.692100 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.692069    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04cdbb97-5fc3-484e-a3e6-1f0f0a208272" (UID: "04cdbb97-5fc3-484e-a3e6-1f0f0a208272"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:21:23.692100 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.692087    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24a7961e-30fa-4382-ab25-5a804afca040" (UID: "24a7961e-30fa-4382-ab25-5a804afca040"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:21:23.694367 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.694340    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr" (OuterVolumeSpecName: "kube-api-access-jcsdr") pod "04cdbb97-5fc3-484e-a3e6-1f0f0a208272" (UID: "04cdbb97-5fc3-484e-a3e6-1f0f0a208272"). InnerVolumeSpecName "kube-api-access-jcsdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:21:23.694514 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.694487    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r" (OuterVolumeSpecName: "kube-api-access-rmp6r") pod "24a7961e-30fa-4382-ab25-5a804afca040" (UID: "24a7961e-30fa-4382-ab25-5a804afca040"). InnerVolumeSpecName "kube-api-access-rmp6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:21:23.694576 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.694494    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04cdbb97-5fc3-484e-a3e6-1f0f0a208272" (UID: "04cdbb97-5fc3-484e-a3e6-1f0f0a208272"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:21:23.793317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.793280    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:23.793317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.793311    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24a7961e-30fa-4382-ab25-5a804afca040-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:23.793317 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.793323    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmp6r\" (UniqueName: \"kubernetes.io/projected/24a7961e-30fa-4382-ab25-5a804afca040-kube-api-access-rmp6r\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:23.793550 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.793331    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcsdr\" (UniqueName: \"kubernetes.io/projected/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-kube-api-access-jcsdr\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:23.793550 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:23.793341    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04cdbb97-5fc3-484e-a3e6-1f0f0a208272-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:24.448076 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.448036    2568 generic.go:356] "Generic (PLEG): container finished" podID="24a7961e-30fa-4382-ab25-5a804afca040" containerID="d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a" exitCode=0
Jul 24 20:21:24.448524 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.448108    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"
Jul 24 20:21:24.448524 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.448124    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" event={"ID":"24a7961e-30fa-4382-ab25-5a804afca040","Type":"ContainerDied","Data":"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"}
Jul 24 20:21:24.448524 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.448177    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl" event={"ID":"24a7961e-30fa-4382-ab25-5a804afca040","Type":"ContainerDied","Data":"6c94420b169b445378d3be0b4b0cc98b9da09dc80981ea8971105f68f75a5c64"}
Jul 24 20:21:24.448524 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.448202    2568 scope.go:117] "RemoveContainer" containerID="d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"
Jul 24 20:21:24.449711 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.449615    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"
Jul 24 20:21:24.449711 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.449617    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf" event={"ID":"04cdbb97-5fc3-484e-a3e6-1f0f0a208272","Type":"ContainerDied","Data":"c25d5e6343059da253f9be3deb658b12b2f1aa4184d2654e881560beb21ebb9b"}
Jul 24 20:21:24.459793 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.459772    2568 scope.go:117] "RemoveContainer" containerID="d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"
Jul 24 20:21:24.460114 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:21:24.460093    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a\": container with ID starting with d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a not found: ID does not exist" containerID="d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"
Jul 24 20:21:24.460181 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.460128    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a"} err="failed to get container status \"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a\": rpc error: code = NotFound desc = could not find container \"d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a\": container with ID starting with d03ada5f68d6b75d3ed0724fa4c45a93ffcf4d878065876409ee391e091d671a not found: ID does not exist"
Jul 24 20:21:24.460181 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.460153    2568 scope.go:117] "RemoveContainer" containerID="2a60b3480547822c4a1c7d3b8ff80a5777206f8c291cc6f37823528745982173"
Jul 24 20:21:24.473066 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.473043    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:21:24.476730 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.476707    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-712eee-d98848d58-l7tnl"]
Jul 24 20:21:24.487792 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.487769    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:21:24.491194 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:24.491169    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-712eee-8688cd4cb6-kwvwf"]
Jul 24 20:21:25.431026 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:25.430991    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" path="/var/lib/kubelet/pods/04cdbb97-5fc3-484e-a3e6-1f0f0a208272/volumes"
Jul 24 20:21:25.431387 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:25.431372    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a7961e-30fa-4382-ab25-5a804afca040" path="/var/lib/kubelet/pods/24a7961e-30fa-4382-ab25-5a804afca040/volumes"
Jul 24 20:21:53.115896 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.115860    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:21:53.116768 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.116736    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" podUID="4737673a-a61d-4b68-83ad-548b4668b91a" containerName="maas-api" containerID="cri-o://64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6" gracePeriod=30
Jul 24 20:21:53.380980 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.380923    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:21:53.476526 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.476488    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5r56\" (UniqueName: \"kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56\") pod \"4737673a-a61d-4b68-83ad-548b4668b91a\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") "
Jul 24 20:21:53.476684 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.476668    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca\") pod \"4737673a-a61d-4b68-83ad-548b4668b91a\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") "
Jul 24 20:21:53.476727 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.476706    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") pod \"4737673a-a61d-4b68-83ad-548b4668b91a\" (UID: \"4737673a-a61d-4b68-83ad-548b4668b91a\") "
Jul 24 20:21:53.477089 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.477048    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4737673a-a61d-4b68-83ad-548b4668b91a" (UID: "4737673a-a61d-4b68-83ad-548b4668b91a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:21:53.478618 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.478590    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56" (OuterVolumeSpecName: "kube-api-access-l5r56") pod "4737673a-a61d-4b68-83ad-548b4668b91a" (UID: "4737673a-a61d-4b68-83ad-548b4668b91a"). InnerVolumeSpecName "kube-api-access-l5r56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:21:53.478754 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.478734    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4737673a-a61d-4b68-83ad-548b4668b91a" (UID: "4737673a-a61d-4b68-83ad-548b4668b91a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:21:53.577689 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.577659    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4737673a-a61d-4b68-83ad-548b4668b91a-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:53.577689 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.577687    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4737673a-a61d-4b68-83ad-548b4668b91a-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:53.577689 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.577696    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5r56\" (UniqueName: \"kubernetes.io/projected/4737673a-a61d-4b68-83ad-548b4668b91a-kube-api-access-l5r56\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:21:53.600103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.600070    2568 generic.go:356] "Generic (PLEG): container finished" podID="4737673a-a61d-4b68-83ad-548b4668b91a" containerID="64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6" exitCode=0
Jul 24 20:21:53.600296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.600111    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"
Jul 24 20:21:53.600296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.600120    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" event={"ID":"4737673a-a61d-4b68-83ad-548b4668b91a","Type":"ContainerDied","Data":"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"}
Jul 24 20:21:53.600296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.600169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk" event={"ID":"4737673a-a61d-4b68-83ad-548b4668b91a","Type":"ContainerDied","Data":"b6f26eca8861a0d024afe9ee98b5dc9c0a7e2fabc7f9a1ddba8605eeb209e630"}
Jul 24 20:21:53.600296 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.600194    2568 scope.go:117] "RemoveContainer" containerID="64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"
Jul 24 20:21:53.609632 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.609611    2568 scope.go:117] "RemoveContainer" containerID="64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"
Jul 24 20:21:53.609900 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:21:53.609882    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6\": container with ID starting with 64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6 not found: ID does not exist" containerID="64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"
Jul 24 20:21:53.609949 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.609908    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6"} err="failed to get container status \"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6\": rpc error: code = NotFound desc = could not find container \"64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6\": container with ID starting with 64288650ed2041504c0594122e8212361a19f31b094049fc98acca4a13cbeba6 not found: ID does not exist"
Jul 24 20:21:53.621770 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.621742    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:21:53.624879 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:53.624858    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-937547-b8ccb9669-wlnfk"]
Jul 24 20:21:55.430922 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:21:55.430883    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4737673a-a61d-4b68-83ad-548b4668b91a" path="/var/lib/kubelet/pods/4737673a-a61d-4b68-83ad-548b4668b91a/volumes"
Jul 24 20:23:02.061516 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.061318    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:23:02.065793 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.061667    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" podUID="96473a99-2788-4678-aaf0-0f45d0eaa97f" containerName="maas-api" containerID="cri-o://25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664" gracePeriod=30
Jul 24 20:23:02.122926 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.122892    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:23:02.123166 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.123119    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" podUID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" containerName="payload-processing" containerID="cri-o://3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c" gracePeriod=30
Jul 24 20:23:02.404874 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.404849    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:23:02.408522 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.408496    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:23:02.526068 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526034    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnr2h\" (UniqueName: \"kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h\") pod \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") "
Jul 24 20:23:02.526319 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526157    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca\") pod \"96473a99-2788-4678-aaf0-0f45d0eaa97f\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") "
Jul 24 20:23:02.526319 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526207    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") pod \"96473a99-2788-4678-aaf0-0f45d0eaa97f\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") "
Jul 24 20:23:02.526319 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526231    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume\") pod \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\" (UID: \"abfcf0c5-5887-42d8-8e5b-57ec483b7205\") "
Jul 24 20:23:02.526319 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526290    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjx5l\" (UniqueName: \"kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l\") pod \"96473a99-2788-4678-aaf0-0f45d0eaa97f\" (UID: \"96473a99-2788-4678-aaf0-0f45d0eaa97f\") "
Jul 24 20:23:02.526645 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526616    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96473a99-2788-4678-aaf0-0f45d0eaa97f" (UID: "96473a99-2788-4678-aaf0-0f45d0eaa97f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:02.526778 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.526637    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abfcf0c5-5887-42d8-8e5b-57ec483b7205" (UID: "abfcf0c5-5887-42d8-8e5b-57ec483b7205"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:02.528295 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.528274    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h" (OuterVolumeSpecName: "kube-api-access-gnr2h") pod "abfcf0c5-5887-42d8-8e5b-57ec483b7205" (UID: "abfcf0c5-5887-42d8-8e5b-57ec483b7205"). InnerVolumeSpecName "kube-api-access-gnr2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:02.528377 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.528305    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l" (OuterVolumeSpecName: "kube-api-access-vjx5l") pod "96473a99-2788-4678-aaf0-0f45d0eaa97f" (UID: "96473a99-2788-4678-aaf0-0f45d0eaa97f"). InnerVolumeSpecName "kube-api-access-vjx5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:02.528424 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.528392    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96473a99-2788-4678-aaf0-0f45d0eaa97f" (UID: "96473a99-2788-4678-aaf0-0f45d0eaa97f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:23:02.628103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.628010    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnr2h\" (UniqueName: \"kubernetes.io/projected/abfcf0c5-5887-42d8-8e5b-57ec483b7205-kube-api-access-gnr2h\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:02.628103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.628048    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96473a99-2788-4678-aaf0-0f45d0eaa97f-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:02.628103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.628083    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96473a99-2788-4678-aaf0-0f45d0eaa97f-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:02.628103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.628095    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abfcf0c5-5887-42d8-8e5b-57ec483b7205-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:02.628103 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.628104    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjx5l\" (UniqueName: \"kubernetes.io/projected/96473a99-2788-4678-aaf0-0f45d0eaa97f-kube-api-access-vjx5l\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:02.952778 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.952739    2568 generic.go:356] "Generic (PLEG): container finished" podID="96473a99-2788-4678-aaf0-0f45d0eaa97f" containerID="25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664" exitCode=0
Jul 24 20:23:02.952961 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.952797    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"
Jul 24 20:23:02.952961 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.952829    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" event={"ID":"96473a99-2788-4678-aaf0-0f45d0eaa97f","Type":"ContainerDied","Data":"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"}
Jul 24 20:23:02.952961 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.952894    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7" event={"ID":"96473a99-2788-4678-aaf0-0f45d0eaa97f","Type":"ContainerDied","Data":"61e089b9faa93bf46d0d524dc143b941020a630a7a193eeaebccbb9b2210b8cc"}
Jul 24 20:23:02.952961 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.952925    2568 scope.go:117] "RemoveContainer" containerID="25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"
Jul 24 20:23:02.954351 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.954327    2568 generic.go:356] "Generic (PLEG): container finished" podID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" containerID="3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c" exitCode=0
Jul 24 20:23:02.954477 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.954361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" event={"ID":"abfcf0c5-5887-42d8-8e5b-57ec483b7205","Type":"ContainerDied","Data":"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"}
Jul 24 20:23:02.954477 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.954393    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"
Jul 24 20:23:02.954477 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.954401    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb" event={"ID":"abfcf0c5-5887-42d8-8e5b-57ec483b7205","Type":"ContainerDied","Data":"c16a6dae0b93ed8eec43f8e736b41a5675c0a8eadbf457cd9b0fc8593856afe9"}
Jul 24 20:23:02.964201 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.964183    2568 scope.go:117] "RemoveContainer" containerID="25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"
Jul 24 20:23:02.964499 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:23:02.964477    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664\": container with ID starting with 25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664 not found: ID does not exist" containerID="25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"
Jul 24 20:23:02.964578 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.964507    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664"} err="failed to get container status \"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664\": rpc error: code = NotFound desc = could not find container \"25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664\": container with ID starting with 25d5f8f1f13ff02ad72516e6ef4655bd5678c3c0ce5591b602bdc6f52f321664 not found: ID does not exist"
Jul 24 20:23:02.964578 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.964524    2568 scope.go:117] "RemoveContainer" containerID="3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"
Jul 24 20:23:02.973694 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.973671    2568 scope.go:117] "RemoveContainer" containerID="3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"
Jul 24 20:23:02.973976 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:23:02.973958    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c\": container with ID starting with 3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c not found: ID does not exist" containerID="3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"
Jul 24 20:23:02.974028 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.973985    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c"} err="failed to get container status \"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c\": rpc error: code = NotFound desc = could not find container \"3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c\": container with ID starting with 3c0352d0c5f5c9f15e030658ff4a18abc61a9d8e7eb8c970685af75f5351fb4c not found: ID does not exist"
Jul 24 20:23:02.977348 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.977322    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:23:02.981431 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.981411    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-307a30-7fd9554655-2j2m7"]
Jul 24 20:23:02.992472 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.992449    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:23:02.996081 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:02.996061    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-307a30-6cc8dccf6d-xjjcb"]
Jul 24 20:23:03.430894 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:03.430800    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96473a99-2788-4678-aaf0-0f45d0eaa97f" path="/var/lib/kubelet/pods/96473a99-2788-4678-aaf0-0f45d0eaa97f/volumes"
Jul 24 20:23:03.431392 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:03.431326    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" path="/var/lib/kubelet/pods/abfcf0c5-5887-42d8-8e5b-57ec483b7205/volumes"
Jul 24 20:23:23.821364 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.821324    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:23:23.821945 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.821567    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" podUID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" containerName="maas-api" containerID="cri-o://0512bf0f9eca2500a6baa720c6ef20207313e189eec9809e63f18bd8d35ee1d9" gracePeriod=30
Jul 24 20:23:23.879046 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.879005    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:23:23.879310 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.879280    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" podUID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" containerName="payload-processing" containerID="cri-o://f0c738fc0d2eeea57cb5bfd2fc32aa47eea5ae2415ebbdbbc72f1739eab824eb" gracePeriod=30
Jul 24 20:23:23.890394 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.890273    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:23:23.890583 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:23.890546    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" podUID="6034a39e-8e89-4d2f-9015-52581aba7f5e" containerName="payload-pre-processing" containerID="cri-o://ccb4dac8796e708695568b755c457b8ba49b128887ca1c64c33e1f3757653c7e" gracePeriod=30
Jul 24 20:23:24.073535 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.073058    2568 generic.go:356] "Generic (PLEG): container finished" podID="6034a39e-8e89-4d2f-9015-52581aba7f5e" containerID="ccb4dac8796e708695568b755c457b8ba49b128887ca1c64c33e1f3757653c7e" exitCode=0
Jul 24 20:23:24.073535 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.073148    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" event={"ID":"6034a39e-8e89-4d2f-9015-52581aba7f5e","Type":"ContainerDied","Data":"ccb4dac8796e708695568b755c457b8ba49b128887ca1c64c33e1f3757653c7e"}
Jul 24 20:23:24.075555 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.075442    2568 generic.go:356] "Generic (PLEG): container finished" podID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" containerID="f0c738fc0d2eeea57cb5bfd2fc32aa47eea5ae2415ebbdbbc72f1739eab824eb" exitCode=0
Jul 24 20:23:24.075555 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.075514    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" event={"ID":"8ac29f63-5da8-4182-af48-dc0dd65fbc55","Type":"ContainerDied","Data":"f0c738fc0d2eeea57cb5bfd2fc32aa47eea5ae2415ebbdbbc72f1739eab824eb"}
Jul 24 20:23:24.077666 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.077476    2568 generic.go:356] "Generic (PLEG): container finished" podID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" containerID="0512bf0f9eca2500a6baa720c6ef20207313e189eec9809e63f18bd8d35ee1d9" exitCode=0
Jul 24 20:23:24.077666 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.077607    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" event={"ID":"74a0f0de-fc1b-46d7-be2a-3a5f87c802da","Type":"ContainerDied","Data":"0512bf0f9eca2500a6baa720c6ef20207313e189eec9809e63f18bd8d35ee1d9"}
Jul 24 20:23:24.165436 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.165408    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:23:24.195753 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.195727    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:23:24.204702 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.204677    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:23:24.225150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225123    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca\") pod \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") "
Jul 24 20:23:24.225309 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225187    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume\") pod \"6034a39e-8e89-4d2f-9015-52581aba7f5e\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") "
Jul 24 20:23:24.225309 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225289    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8vld\" (UniqueName: \"kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld\") pod \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") "
Jul 24 20:23:24.225439 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225322    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") pod \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") "
Jul 24 20:23:24.225439 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htwcj\" (UniqueName: \"kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj\") pod \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\" (UID: \"74a0f0de-fc1b-46d7-be2a-3a5f87c802da\") "
Jul 24 20:23:24.225439 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225426    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z67zn\" (UniqueName: \"kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn\") pod \"6034a39e-8e89-4d2f-9015-52581aba7f5e\" (UID: \"6034a39e-8e89-4d2f-9015-52581aba7f5e\") "
Jul 24 20:23:24.225600 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225466    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume\") pod \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\" (UID: \"8ac29f63-5da8-4182-af48-dc0dd65fbc55\") "
Jul 24 20:23:24.225600 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225506    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74a0f0de-fc1b-46d7-be2a-3a5f87c802da" (UID: "74a0f0de-fc1b-46d7-be2a-3a5f87c802da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:24.225600 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225540    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6034a39e-8e89-4d2f-9015-52581aba7f5e" (UID: "6034a39e-8e89-4d2f-9015-52581aba7f5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:24.226152 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225771    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.226152 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225792    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6034a39e-8e89-4d2f-9015-52581aba7f5e-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.226152 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.225989    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ac29f63-5da8-4182-af48-dc0dd65fbc55" (UID: "8ac29f63-5da8-4182-af48-dc0dd65fbc55"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:24.228190 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.228138    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn" (OuterVolumeSpecName: "kube-api-access-z67zn") pod "6034a39e-8e89-4d2f-9015-52581aba7f5e" (UID: "6034a39e-8e89-4d2f-9015-52581aba7f5e"). InnerVolumeSpecName "kube-api-access-z67zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:24.228965 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.228935    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld" (OuterVolumeSpecName: "kube-api-access-g8vld") pod "8ac29f63-5da8-4182-af48-dc0dd65fbc55" (UID: "8ac29f63-5da8-4182-af48-dc0dd65fbc55"). InnerVolumeSpecName "kube-api-access-g8vld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:24.229136 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.229103    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj" (OuterVolumeSpecName: "kube-api-access-htwcj") pod "74a0f0de-fc1b-46d7-be2a-3a5f87c802da" (UID: "74a0f0de-fc1b-46d7-be2a-3a5f87c802da"). InnerVolumeSpecName "kube-api-access-htwcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:24.229253 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.229158    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74a0f0de-fc1b-46d7-be2a-3a5f87c802da" (UID: "74a0f0de-fc1b-46d7-be2a-3a5f87c802da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:23:24.326657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.326571    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htwcj\" (UniqueName: \"kubernetes.io/projected/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-kube-api-access-htwcj\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.326657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.326610    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z67zn\" (UniqueName: \"kubernetes.io/projected/6034a39e-8e89-4d2f-9015-52581aba7f5e-kube-api-access-z67zn\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.326657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.326625    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac29f63-5da8-4182-af48-dc0dd65fbc55-plugins-config-volume\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.326657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.326639    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8vld\" (UniqueName: \"kubernetes.io/projected/8ac29f63-5da8-4182-af48-dc0dd65fbc55-kube-api-access-g8vld\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:24.326657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:24.326651    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74a0f0de-fc1b-46d7-be2a-3a5f87c802da-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:25.084702 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.084668    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"
Jul 24 20:23:25.085178 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.084667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr" event={"ID":"6034a39e-8e89-4d2f-9015-52581aba7f5e","Type":"ContainerDied","Data":"bee8be3648d5bee9de1f96db1f5c381a89ee457f26b333fa7c64b5121e346e60"}
Jul 24 20:23:25.085178 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.084799    2568 scope.go:117] "RemoveContainer" containerID="ccb4dac8796e708695568b755c457b8ba49b128887ca1c64c33e1f3757653c7e"
Jul 24 20:23:25.086219 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.085968    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh" event={"ID":"8ac29f63-5da8-4182-af48-dc0dd65fbc55","Type":"ContainerDied","Data":"9baafd6a4d6ade7684dc5df35289def1171b16f7474f663ff549217f8c96310b"}
Jul 24 20:23:25.086219 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.085978    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"
Jul 24 20:23:25.087606 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.087579    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"
Jul 24 20:23:25.087717 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.087620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm" event={"ID":"74a0f0de-fc1b-46d7-be2a-3a5f87c802da","Type":"ContainerDied","Data":"68d209ec5a403f543e4b7a6761745d493c26fcf476795c9c59fe281cc82cfb72"}
Jul 24 20:23:25.094769 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.094740    2568 scope.go:117] "RemoveContainer" containerID="f0c738fc0d2eeea57cb5bfd2fc32aa47eea5ae2415ebbdbbc72f1739eab824eb"
Jul 24 20:23:25.105192 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.105170    2568 scope.go:117] "RemoveContainer" containerID="0512bf0f9eca2500a6baa720c6ef20207313e189eec9809e63f18bd8d35ee1d9"
Jul 24 20:23:25.115178 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.115143    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:23:25.122957 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.122932    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-53d0a0-64cb56b656-8h4cr"]
Jul 24 20:23:25.134871 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.134847    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:23:25.141134 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.141112    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-53d0a0-6d57cc7c5-4qvjm"]
Jul 24 20:23:25.153519 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.153496    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:23:25.157657 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.157632    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-53d0a0-58866b6754-2tqfh"]
Jul 24 20:23:25.431236 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.431149    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6034a39e-8e89-4d2f-9015-52581aba7f5e" path="/var/lib/kubelet/pods/6034a39e-8e89-4d2f-9015-52581aba7f5e/volumes"
Jul 24 20:23:25.431675 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.431654    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" path="/var/lib/kubelet/pods/74a0f0de-fc1b-46d7-be2a-3a5f87c802da/volumes"
Jul 24 20:23:25.432034 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:25.432016    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" path="/var/lib/kubelet/pods/8ac29f63-5da8-4182-af48-dc0dd65fbc55/volumes"
Jul 24 20:23:45.836312 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:45.836166    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:23:45.836814 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:45.836488    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" podUID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" containerName="maas-api" containerID="cri-o://8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10" gracePeriod=30
Jul 24 20:23:46.094588 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.094531    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:23:46.194449 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.194418    2568 generic.go:356] "Generic (PLEG): container finished" podID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" containerID="8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10" exitCode=0
Jul 24 20:23:46.194628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.194470    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"
Jul 24 20:23:46.194628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.194485    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" event={"ID":"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04","Type":"ContainerDied","Data":"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"}
Jul 24 20:23:46.194628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.194527    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc" event={"ID":"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04","Type":"ContainerDied","Data":"e2fa8328088096b4b8adbada337c02dd779b920035de4bd7e020dd80e376ddc6"}
Jul 24 20:23:46.194628 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.194558    2568 scope.go:117] "RemoveContainer" containerID="8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"
Jul 24 20:23:46.203158 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.203136    2568 scope.go:117] "RemoveContainer" containerID="8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"
Jul 24 20:23:46.203466 ip-10-0-129-222 kubenswrapper[2568]: E0724 20:23:46.203440    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10\": container with ID starting with 8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10 not found: ID does not exist" containerID="8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"
Jul 24 20:23:46.203532 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.203479    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10"} err="failed to get container status \"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10\": rpc error: code = NotFound desc = could not find container \"8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10\": container with ID starting with 8abb7ad8580ee84fb0d5a211f3f276d6c4f4c48cfee3ff61fc2b7b58f9a02f10 not found: ID does not exist"
Jul 24 20:23:46.231786 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.231747    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca\") pod \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") "
Jul 24 20:23:46.231912 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.231809    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6xrw\" (UniqueName: \"kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw\") pod \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") "
Jul 24 20:23:46.231912 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.231899    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls\") pod \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\" (UID: \"1d5eb3ad-5628-4490-8b90-d5c12a9c7d04\") "
Jul 24 20:23:46.232092 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.232069    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" (UID: "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:23:46.232152 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.232139    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-openshift-service-ca\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:46.233843 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.233821    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" (UID: "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:23:46.233958 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.233942    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw" (OuterVolumeSpecName: "kube-api-access-l6xrw") pod "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" (UID: "1d5eb3ad-5628-4490-8b90-d5c12a9c7d04"). InnerVolumeSpecName "kube-api-access-l6xrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:23:46.333268 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.333218    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6xrw\" (UniqueName: \"kubernetes.io/projected/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-kube-api-access-l6xrw\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:46.333433 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.333286    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04-maas-api-tls\") on node \"ip-10-0-129-222.ec2.internal\" DevicePath \"\""
Jul 24 20:23:46.518863 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.518832    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:23:46.522222 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:46.522195    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-3f5fc9-b45c98d67-8mnrc"]
Jul 24 20:23:47.430267 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:47.430222    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" path="/var/lib/kubelet/pods/1d5eb3ad-5628-4490-8b90-d5c12a9c7d04/volumes"
Jul 24 20:23:57.348728 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:23:57.348630    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7779cf7f79-bbcbd_1715dc07-fed9-4618-8ba0-8c126375bf8b/authorino/0.log"
Jul 24 20:24:02.657984 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:02.657939    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w4jz8_f05f5dd3-3210-416e-ae8f-844ba4e45080/manager/0.log"
Jul 24 20:24:02.779440 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:02.779404    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zbmdv_ddcd5f0d-269b-4420-af46-8e88c2329ee8/manager/0.log"
Jul 24 20:24:02.902984 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:02.902951    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c9fc846b-4cqls_a263a1e2-8a11-449c-95c2-62789338ed07/manager/0.log"
Jul 24 20:24:03.016086 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:03.016059    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-px656_e7a783e4-94d5-4d79-988f-1a2cad8ab410/server/0.log"
Jul 24 20:24:03.133607 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:03.133578    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jbbt8_86d1b4ca-93ac-44e0-b322-18f5bad6ffc4/manager/0.log"
Jul 24 20:24:03.243966 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:03.243938    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d7d8f86c4-6n4tp_97897029-4b76-40d5-b09b-679295507f40/copy-manifests/0.log"
Jul 24 20:24:03.261763 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:03.261717    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d7d8f86c4-6n4tp_97897029-4b76-40d5-b09b-679295507f40/manager/0.log"
Jul 24 20:24:03.858904 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:03.858862    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-864cf74d58-49l72_509e08a6-35e6-43a1-9649-df8d5bf9ecdb/maas-api/0.log"
Jul 24 20:24:04.302029 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:04.301995    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wnvrp_d98cdbf6-fe6a-43bc-87f9-ceeed9e0649b/postgres/0.log"
Jul 24 20:24:05.550311 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:05.550278    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7779cf7f79-bbcbd_1715dc07-fed9-4618-8ba0-8c126375bf8b/authorino/0.log"
Jul 24 20:24:05.665839 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:05.665804    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cwzs6_15c8e78c-5673-45a7-880c-3d0884d2610d/manager/0.log"
Jul 24 20:24:05.773504 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:05.773469    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k9wqc_2e282371-e8d2-46a5-8a23-3bab91d49af4/manager/0.log"
Jul 24 20:24:06.014927 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:06.014880    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jp5r8_811a2504-569a-4559-923a-d0a496f6e501/registry-server/0.log"
Jul 24 20:24:06.138211 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:06.138166    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mz8px_686afa03-4122-4c5c-89f3-0baa54f57126/manager/0.log"
Jul 24 20:24:06.360892 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:06.360812    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h9njb_b4f2f490-2b44-42a3-a21a-173602782e30/manager/0.log"
Jul 24 20:24:06.925007 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:06.924971    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c49bfc89-66vph_27ce4ae1-2cc4-4471-9eb5-1eef532168b0/kube-auth-proxy/0.log"
Jul 24 20:24:07.269763 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:07.269721    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-24tb9_634e271f-d2bf-40bc-ba90-828a063c25f8/payload-pre-processing/0.log"
Jul 24 20:24:07.951395 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:07.951361    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-l8cvl_080695bd-b429-4e83-921a-504db28b663c/main/0.log"
Jul 24 20:24:08.408646 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:08.408555    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-vx56r_d1fccd45-8d5f-494b-9761-33683ee72078/main/0.log"
Jul 24 20:24:13.965646 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.965614    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzgtx/must-gather-4pghh"]
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.965979    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6034a39e-8e89-4d2f-9015-52581aba7f5e" containerName="payload-pre-processing"
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.965991    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6034a39e-8e89-4d2f-9015-52581aba7f5e" containerName="payload-pre-processing"
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966000    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" containerName="maas-api"
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966006    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" containerName="maas-api"
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966019    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" containerName="payload-processing"
Jul 24 20:24:13.966031 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966025    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966042    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966048    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966054    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4737673a-a61d-4b68-83ad-548b4668b91a" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966060    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4737673a-a61d-4b68-83ad-548b4668b91a" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966066    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96473a99-2788-4678-aaf0-0f45d0eaa97f" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966071    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="96473a99-2788-4678-aaf0-0f45d0eaa97f" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966077    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a7961e-30fa-4382-ab25-5a804afca040" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966082    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a7961e-30fa-4382-ab25-5a804afca040" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966090    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966095    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966101    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966105    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966158    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6034a39e-8e89-4d2f-9015-52581aba7f5e" containerName="payload-pre-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966164    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4737673a-a61d-4b68-83ad-548b4668b91a" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966169    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ac29f63-5da8-4182-af48-dc0dd65fbc55" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966174    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="96473a99-2788-4678-aaf0-0f45d0eaa97f" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966179    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="04cdbb97-5fc3-484e-a3e6-1f0f0a208272" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966185    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a7961e-30fa-4382-ab25-5a804afca040" containerName="payload-processing"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966191    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5eb3ad-5628-4490-8b90-d5c12a9c7d04" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966199    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="74a0f0de-fc1b-46d7-be2a-3a5f87c802da" containerName="maas-api"
Jul 24 20:24:13.966224 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.966204    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfcf0c5-5887-42d8-8e5b-57ec483b7205" containerName="payload-processing"
Jul 24 20:24:13.969571 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.969549    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:13.972867 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.972846    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzgtx\"/\"kube-root-ca.crt\""
Jul 24 20:24:13.973863 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.973845    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gzgtx\"/\"default-dockercfg-q9hgm\""
Jul 24 20:24:13.973945 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.973871    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzgtx\"/\"openshift-service-ca.crt\""
Jul 24 20:24:13.977448 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:13.977423    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzgtx/must-gather-4pghh"]
Jul 24 20:24:14.088950 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.088921    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8dd3072-29ad-42c4-815a-ca330ec0dab2-must-gather-output\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.089123 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.088995    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlqlj\" (UniqueName: \"kubernetes.io/projected/d8dd3072-29ad-42c4-815a-ca330ec0dab2-kube-api-access-wlqlj\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.190033 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.189996    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlqlj\" (UniqueName: \"kubernetes.io/projected/d8dd3072-29ad-42c4-815a-ca330ec0dab2-kube-api-access-wlqlj\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.190177 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.190053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8dd3072-29ad-42c4-815a-ca330ec0dab2-must-gather-output\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.190389 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.190372    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d8dd3072-29ad-42c4-815a-ca330ec0dab2-must-gather-output\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.201264 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.201237    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlqlj\" (UniqueName: \"kubernetes.io/projected/d8dd3072-29ad-42c4-815a-ca330ec0dab2-kube-api-access-wlqlj\") pod \"must-gather-4pghh\" (UID: \"d8dd3072-29ad-42c4-815a-ca330ec0dab2\") " pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.279722 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.279643    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzgtx/must-gather-4pghh"
Jul 24 20:24:14.419322 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.419290    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzgtx/must-gather-4pghh"]
Jul 24 20:24:14.421871 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:24:14.421846    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8dd3072_29ad_42c4_815a_ca330ec0dab2.slice/crio-4cc8c8794fcd82e883485b1a65db0f0e50911701a3d515a7766c4edb8ec2636d WatchSource:0}: Error finding container 4cc8c8794fcd82e883485b1a65db0f0e50911701a3d515a7766c4edb8ec2636d: Status 404 returned error can't find the container with id 4cc8c8794fcd82e883485b1a65db0f0e50911701a3d515a7766c4edb8ec2636d
Jul 24 20:24:14.423907 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:14.423890    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:24:15.338998 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:15.338962    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzgtx/must-gather-4pghh" event={"ID":"d8dd3072-29ad-42c4-815a-ca330ec0dab2","Type":"ContainerStarted","Data":"4cc8c8794fcd82e883485b1a65db0f0e50911701a3d515a7766c4edb8ec2636d"}
Jul 24 20:24:16.348513 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:16.348238    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzgtx/must-gather-4pghh" event={"ID":"d8dd3072-29ad-42c4-815a-ca330ec0dab2","Type":"ContainerStarted","Data":"8ac80f7128872ee311aebb6cc6dc3c57266360ecb837457b81060b7dae5c9f43"}
Jul 24 20:24:16.348513 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:16.348315    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzgtx/must-gather-4pghh" event={"ID":"d8dd3072-29ad-42c4-815a-ca330ec0dab2","Type":"ContainerStarted","Data":"b04b68017d9f89dd616cf97aafbbd735096c4c2c8c934ac42c7b7882b1741949"}
Jul 24 20:24:16.366150 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:16.366096    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gzgtx/must-gather-4pghh" podStartSLOduration=2.492453943 podStartE2EDuration="3.36608216s" totalImagesPullingTime="873.628217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:24:16.363929672 +0000 UTC m=+2987.458475291" watchObservedRunningTime="2026-07-24 20:24:16.36608216 +0000 UTC m=+2987.460627772"
Jul 24 20:24:16.901068 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:16.901024    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pczcr_f06a533e-1542-4c4d-aa64-7a566d7ab0ad/global-pull-secret-syncer/0.log"
Jul 24 20:24:16.990545 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:16.990509    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rbnml_48bf3fb5-7dc5-430e-aaa6-d1cb35cac8cb/konnectivity-agent/0.log"
Jul 24 20:24:17.042835 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:17.042804    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-222.ec2.internal_e46ae8c13e8686631be2da87086a49c5/haproxy/0.log"
Jul 24 20:24:22.681613 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:22.681577    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7779cf7f79-bbcbd_1715dc07-fed9-4618-8ba0-8c126375bf8b/authorino/0.log"
Jul 24 20:24:22.713910 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:22.713879    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cwzs6_15c8e78c-5673-45a7-880c-3d0884d2610d/manager/0.log"
Jul 24 20:24:22.748646 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:22.748607    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-k9wqc_2e282371-e8d2-46a5-8a23-3bab91d49af4/manager/0.log"
Jul 24 20:24:22.836241 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:22.836199    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jp5r8_811a2504-569a-4559-923a-d0a496f6e501/registry-server/0.log"
Jul 24 20:24:23.033089 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:23.033032    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mz8px_686afa03-4122-4c5c-89f3-0baa54f57126/manager/0.log"
Jul 24 20:24:23.182163 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:23.182112    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h9njb_b4f2f490-2b44-42a3-a21a-173602782e30/manager/0.log"
Jul 24 20:24:25.078703 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:25.078663    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-q9jjg_3ba1fbf3-0dc3-4aa6-be2f-ef5ed4fb052a/cluster-monitoring-operator/0.log"
Jul 24 20:24:25.274515 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:25.274482    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/node-exporter/0.log"
Jul 24 20:24:25.295962 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:25.295911    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/kube-rbac-proxy/0.log"
Jul 24 20:24:25.319052 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:25.319019    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9mfm_a69f0e14-d134-4c35-8ab8-b730dde3913d/init-textfile/0.log"
Jul 24 20:24:26.822965 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.822914    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"]
Jul 24 20:24:26.828164 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.828136    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:26.836482 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.836447    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"]
Jul 24 20:24:26.928774 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.928677    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-lib-modules\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:26.928958 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.928837    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-podres\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:26.928958 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.928870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-sys\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:26.928958 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.928936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg87m\" (UniqueName: \"kubernetes.io/projected/a4aa5505-9b45-4c25-9730-d47ed18d6074-kube-api-access-kg87m\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:26.929144 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:26.929114    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-proc\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030452 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-lib-modules\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030632 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-podres\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030632 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030494    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-sys\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030632 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030527    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg87m\" (UniqueName: \"kubernetes.io/projected/a4aa5505-9b45-4c25-9730-d47ed18d6074-kube-api-access-kg87m\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030632 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-proc\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030640    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-podres\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030647    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-lib-modules\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030703    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-proc\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.030808 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.030704    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4aa5505-9b45-4c25-9730-d47ed18d6074-sys\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.040334 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.040298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg87m\" (UniqueName: \"kubernetes.io/projected/a4aa5505-9b45-4c25-9730-d47ed18d6074-kube-api-access-kg87m\") pod \"perf-node-gather-daemonset-dhrpk\" (UID: \"a4aa5505-9b45-4c25-9730-d47ed18d6074\") " pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.143750 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.143430    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:27.344734 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.343097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"]
Jul 24 20:24:27.345386 ip-10-0-129-222 kubenswrapper[2568]: W0724 20:24:27.345354    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda4aa5505_9b45_4c25_9730_d47ed18d6074.slice/crio-783e60d31a77045cecff9131b3f33ed33674081d06b097fa7190ae79ce40ec16 WatchSource:0}: Error finding container 783e60d31a77045cecff9131b3f33ed33674081d06b097fa7190ae79ce40ec16: Status 404 returned error can't find the container with id 783e60d31a77045cecff9131b3f33ed33674081d06b097fa7190ae79ce40ec16
Jul 24 20:24:27.433067 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.433011    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk" event={"ID":"a4aa5505-9b45-4c25-9730-d47ed18d6074","Type":"ContainerStarted","Data":"783e60d31a77045cecff9131b3f33ed33674081d06b097fa7190ae79ce40ec16"}
Jul 24 20:24:27.710136 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.710107    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:24:27.718536 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:27.718504    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/2.log"
Jul 24 20:24:28.204154 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:28.204113    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-s2l2l_2d980ff5-4ea0-4aa5-a12c-b9c444e7b976/download-server/0.log"
Jul 24 20:24:28.438167 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:28.438126    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk" event={"ID":"a4aa5505-9b45-4c25-9730-d47ed18d6074","Type":"ContainerStarted","Data":"082d45d1da92b4ec2715269f8dd12ac40b980507e2611340a6dd343694805581"}
Jul 24 20:24:28.439035 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:28.439004    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:28.455804 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:28.455695    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk" podStartSLOduration=2.455674161 podStartE2EDuration="2.455674161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:24:28.454281345 +0000 UTC m=+2999.548826956" watchObservedRunningTime="2026-07-24 20:24:28.455674161 +0000 UTC m=+2999.550219775"
Jul 24 20:24:28.735578 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:28.735479    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-65zwf_73508393-3f14-4717-98b4-a2967ce4762a/volume-data-source-validator/0.log"
Jul 24 20:24:29.584820 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:29.584785    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:24:29.591749 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:29.591712    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-22sns_189ab177-f6fb-41e9-adb2-9a1340838434/console-operator/1.log"
Jul 24 20:24:29.749392 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:29.749366    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cnzsr_d609910f-0265-420c-92db-0a74114ffb68/dns/0.log"
Jul 24 20:24:29.768510 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:29.768479    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cnzsr_d609910f-0265-420c-92db-0a74114ffb68/kube-rbac-proxy/0.log"
Jul 24 20:24:29.906713 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:29.906641    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9dtbm_086cfcea-8454-495a-8fac-bf098927b30d/dns-node-resolver/0.log"
Jul 24 20:24:30.490387 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:30.490353    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bf74b7f4b-sw28q_7260b00e-cf5d-48ae-b928-ca2b9901b97a/registry/0.log"
Jul 24 20:24:30.552278 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:30.552220    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmnn5_233beb3e-a78b-411a-bdd0-20a9caf4dc22/node-ca/0.log"
Jul 24 20:24:31.632137 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:31.632110    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c49bfc89-66vph_27ce4ae1-2cc4-4471-9eb5-1eef532168b0/kube-auth-proxy/0.log"
Jul 24 20:24:31.751793 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:31.751752    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-24tb9_634e271f-d2bf-40bc-ba90-828a063c25f8/payload-pre-processing/0.log"
Jul 24 20:24:32.608181 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:32.608145    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hp5zw_e5f5ca97-0438-46ca-98b3-89481ef9e461/serve-healthcheck-canary/0.log"
Jul 24 20:24:33.114141 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:33.114105    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jqsvf_5ba3423a-ccd8-4583-b983-fcc2e041e4aa/insights-operator/0.log"
Jul 24 20:24:33.117033 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:33.117000    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-jqsvf_5ba3423a-ccd8-4583-b983-fcc2e041e4aa/insights-operator/1.log"
Jul 24 20:24:33.262146 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:33.262115    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvz4h_0207c691-47cf-482e-bb94-7c452d36481d/kube-rbac-proxy/0.log"
Jul 24 20:24:33.281369 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:33.281336    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvz4h_0207c691-47cf-482e-bb94-7c452d36481d/exporter/0.log"
Jul 24 20:24:33.301033 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:33.300989    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvz4h_0207c691-47cf-482e-bb94-7c452d36481d/extractor/0.log"
Jul 24 20:24:35.339278 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:35.339229    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w4jz8_f05f5dd3-3210-416e-ae8f-844ba4e45080/manager/0.log"
Jul 24 20:24:35.467576 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:35.467542    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gzgtx/perf-node-gather-daemonset-dhrpk"
Jul 24 20:24:35.745856 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:35.745751    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zbmdv_ddcd5f0d-269b-4420-af46-8e88c2329ee8/manager/0.log"
Jul 24 20:24:35.990876 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:35.990846    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c9fc846b-4cqls_a263a1e2-8a11-449c-95c2-62789338ed07/manager/0.log"
Jul 24 20:24:36.009455 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:36.009333    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-px656_e7a783e4-94d5-4d79-988f-1a2cad8ab410/server/0.log"
Jul 24 20:24:36.153561 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:36.153509    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jbbt8_86d1b4ca-93ac-44e0-b322-18f5bad6ffc4/manager/0.log"
Jul 24 20:24:36.328514 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:36.328396    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d7d8f86c4-6n4tp_97897029-4b76-40d5-b09b-679295507f40/manager/0.log"
Jul 24 20:24:36.347291 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:36.347241    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d7d8f86c4-6n4tp_97897029-4b76-40d5-b09b-679295507f40/copy-manifests/0.log"
Jul 24 20:24:38.077947 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:38.077915    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c49bfc89-66vph_27ce4ae1-2cc4-4471-9eb5-1eef532168b0/kube-auth-proxy/0.log"
Jul 24 20:24:38.173610 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:38.173575    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-24tb9_634e271f-d2bf-40bc-ba90-828a063c25f8/payload-pre-processing/0.log"
Jul 24 20:24:39.691424 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:39.691393    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85c49bfc89-66vph_27ce4ae1-2cc4-4471-9eb5-1eef532168b0/kube-auth-proxy/0.log"
Jul 24 20:24:39.783707 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:39.783671    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-24tb9_634e271f-d2bf-40bc-ba90-828a063c25f8/payload-pre-processing/0.log"
Jul 24 20:24:44.410694 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.410662    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/kube-multus-additional-cni-plugins/0.log"
Jul 24 20:24:44.428003 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.427975    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/egress-router-binary-copy/0.log"
Jul 24 20:24:44.444552 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.444516    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/cni-plugins/0.log"
Jul 24 20:24:44.464034 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.464008    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/bond-cni-plugin/0.log"
Jul 24 20:24:44.491275 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.491203    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/routeoverride-cni/0.log"
Jul 24 20:24:44.511491 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.511470    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/whereabouts-cni-bincopy/0.log"
Jul 24 20:24:44.530418 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.530382    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxrpz_a12ce0e4-b9cf-4987-97a4-25fbcfccc79e/whereabouts-cni/0.log"
Jul 24 20:24:44.762718 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.762675    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ndh4r_41051d40-c8a0-4cc9-8a9a-a1a69dbbbe2f/kube-multus/0.log"
Jul 24 20:24:44.786173 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.786141    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8mr59_52a08db7-be3f-4307-a4eb-be1869fcd7b9/network-metrics-daemon/0.log"
Jul 24 20:24:44.800230 ip-10-0-129-222 kubenswrapper[2568]: I0724 20:24:44.800198    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8mr59_52a08db7-be3f-4307-a4eb-be1869fcd7b9/kube-rbac-proxy/0.log"